{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  第3章 k近邻法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1．$k$近邻法是基本且简单的分类与回归方法。$k$近邻法的基本做法是：对给定的训练实例点和输入实例点，首先确定输入实例点的$k$个最近邻训练实例点，然后利用这$k$个训练实例点的类的多数来预测输入实例点的类。\n",
    "\n",
    "2．$k$近邻模型对应于基于训练数据集对特征空间的一个划分。$k$近邻法中，当训练集、距离度量、$k$值及分类决策规则确定后，其结果唯一确定。\n",
    "\n",
    "3．$k$近邻法三要素：距离度量、$k$值的选择和分类决策规则。常用的距离度量是欧氏距离及更一般的**pL**距离。$k$值小时，$k$近邻模型更复杂；$k$值大时，$k$近邻模型更简单。$k$值的选择反映了对近似误差与估计误差之间的权衡，通常由交叉验证选择最优的$k$。\n",
    "\n",
    "常用的分类决策规则是多数表决，对应于经验风险最小化。\n",
    "\n",
    "4．$k$近邻法的实现需要考虑如何快速搜索k个最近邻点。**kd**树是一种便于对k维空间中的数据进行快速检索的数据结构。kd树是二叉树，表示对$k$维空间的一个划分，其每个结点对应于$k$维空间划分中的一个超矩形区域。利用**kd**树可以省去对大部分数据点的搜索， 从而减少搜索的计算量。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 距离度量"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "设特征空间$x$是$n$维实数向量空间 ，$x_{i}, x_{j} \\in \\mathcal{X}$,$x_{i}=\\left(x_{i}^{(1)}, x_{i}^{(2)}, \\cdots, x_{i}^{(n)}\\right)^{\\mathrm{T}}$,$x_{j}=\\left(x_{j}^{(1)}, x_{j}^{(2)}, \\cdots, x_{j}^{(n)}\\right)^{\\mathrm{T}}$\n",
    "，则：$x_i$,$x_j$的$L_p$距离定义为:\n",
    "\n",
    "\n",
    "$L_{p}\\left(x_{i}, x_{j}\\right)=\\left(\\sum_{i=1}^{n}\\left|x_{i}^{(i)}-x_{j}^{(l)}\\right|^{p}\\right)^{\\frac{1}{p}}$\n",
    "\n",
    "- $p= 1$  曼哈顿距离\n",
    "- $p= 2$  欧氏距离\n",
    "- $p= \\infty$   切比雪夫距离"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "from itertools import combinations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def L(x, y, p=2):\n",
    "    # x1 = [1, 1], x2 = [5,1]\n",
    "    if len(x) == len(y) and len(x) > 1:\n",
    "        sum = 0\n",
    "        for i in range(len(x)):\n",
    "            sum += math.pow(abs(x[i] - y[i]), p)\n",
    "        return math.pow(sum, 1 / p)\n",
    "    else:\n",
    "        return 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 课本例3.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "x1 = [1, 1]\n",
    "x2 = [5, 1]\n",
    "x3 = [4, 4]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "(4.0, '1-[5, 1]')\n(4.0, '1-[5, 1]')\n(3.7797631496846193, '1-[4, 4]')\n(3.5676213450081633, '1-[4, 4]')\n"
    }
   ],
   "source": [
    "# x1, x2\n",
    "for i in range(1, 5):\n",
    "    r = {'1-{}'.format(c): L(x1, c, p=i) for c in [x2, x3]}\n",
    "    print(min(zip(r.values(), r.keys())))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "python实现，遍历所有数据点，找出$n$个距离最近的点的分类情况，少数服从多数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "from sklearn.datasets import load_iris\n",
    "from sklearn.model_selection import train_test_split\n",
    "from collections import Counter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# data\n",
    "iris = load_iris()\n",
    "df = pd.DataFrame(iris.data, columns=iris.feature_names)\n",
    "df['label'] = iris.target\n",
    "df.columns = ['sepal length', 'sepal width', 'petal length', 'petal width', 'label']\n",
    "# data = np.array(df.iloc[:100, [0, 1, -1]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "     sepal length  sepal width  petal length  petal width  label\n0             5.1          3.5           1.4          0.2      0\n1             4.9          3.0           1.4          0.2      0\n2             4.7          3.2           1.3          0.2      0\n3             4.6          3.1           1.5          0.2      0\n4             5.0          3.6           1.4          0.2      0\n..            ...          ...           ...          ...    ...\n145           6.7          3.0           5.2          2.3      2\n146           6.3          2.5           5.0          1.9      2\n147           6.5          3.0           5.2          2.0      2\n148           6.2          3.4           5.4          2.3      2\n149           5.9          3.0           5.1          1.8      2\n\n[150 rows x 5 columns]",
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>sepal length</th>\n      <th>sepal width</th>\n      <th>petal length</th>\n      <th>petal width</th>\n      <th>label</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>5.1</td>\n      <td>3.5</td>\n      <td>1.4</td>\n      <td>0.2</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>4.9</td>\n      <td>3.0</td>\n      <td>1.4</td>\n      <td>0.2</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>4.7</td>\n      <td>3.2</td>\n      <td>1.3</td>\n      <td>0.2</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4.6</td>\n      <td>3.1</td>\n      <td>1.5</td>\n      <td>0.2</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>5.0</td>\n      <td>3.6</td>\n      <td>1.4</td>\n      <td>0.2</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>...</th>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n      <td>...</td>\n    </tr>\n    <tr>\n      <th>145</th>\n      <td>6.7</td>\n      <td>3.0</td>\n      <td>5.2</td>\n      <td>2.3</td>\n      <td>2</td>\n    </tr>\n    <tr>\n      <th>146</th>\n      <td>6.3</td>\n      <td>2.5</td>\n      <td>5.0</td>\n      <td>1.9</td>\n      <td>2</td>\n    </tr>\n    <tr>\n      <th>147</th>\n      <td>6.5</td>\n      <td>3.0</td>\n      <td>5.2</td>\n      <td>2.0</td>\n      <td>2</td>\n    </tr>\n    <tr>\n      <th>148</th>\n      <td>6.2</td>\n      <td>3.4</td>\n      <td>5.4</td>\n      <td>2.3</td>\n      <td>2</td>\n    </tr>\n    <tr>\n      <th>149</th>\n      <td>5.9</td>\n      <td>3.0</td>\n      <td>5.1</td>\n      <td>1.8</td>\n      <td>2</td>\n    </tr>\n  </tbody>\n</table>\n<p>150 rows × 5 columns</p>\n</div>"
     },
     "metadata": {},
     "execution_count": 7
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "<matplotlib.legend.Legend at 0x1e1b750bd48>"
     },
     "metadata": {},
     "execution_count": 8
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": "<Figure size 432x288 with 1 Axes>",
      "image/svg+xml": "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\r\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\r\n  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\r\n<!-- Created with matplotlib (https://matplotlib.org/) -->\r\n<svg height=\"264.348196pt\" version=\"1.1\" viewBox=\"0 0 385.78125 264.348196\" width=\"385.78125pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\r\n <defs>\r\n  <style type=\"text/css\">\r\n*{stroke-linecap:butt;stroke-linejoin:round;}\r\n  </style>\r\n </defs>\r\n <g id=\"figure_1\">\r\n  <g id=\"patch_1\">\r\n   <path d=\"M 0 264.348196 \r\nL 385.78125 264.348196 \r\nL 385.78125 0 \r\nL 0 0 \r\nz\r\n\" style=\"fill:none;\"/>\r\n  </g>\r\n  <g id=\"axes_1\">\r\n   <g id=\"patch_2\">\r\n    <path d=\"M 43.78125 226.791946 \r\nL 378.58125 226.791946 \r\nL 378.58125 9.351946 \r\nL 43.78125 9.351946 \r\nz\r\n\" style=\"fill:#ffffff;\"/>\r\n   </g>\r\n   <g id=\"PathCollection_1\">\r\n    <defs>\r\n     <path d=\"M 0 3 \r\nC 0.795609 3 1.55874 2.683901 2.12132 2.12132 \r\nC 2.683901 1.55874 3 0.795609 3 0 \r\nC 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \r\nC 1.55874 -2.683901 0.795609 -3 0 -3 \r\nC -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \r\nC -2.683901 -1.55874 -3 -0.795609 -3 0 \r\nC -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \r\nC -1.55874 2.683901 -0.795609 3 0 3 \r\nz\r\n\" id=\"m488bef92e8\" style=\"stroke:#1f77b4;\"/>\r\n    </defs>\r\n    <g clip-path=\"url(#pa6da11bef6)\">\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"149.18125\" xlink:href=\"#m488bef92e8\" y=\"93.362855\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"126.635795\" xlink:href=\"#m488bef92e8\" y=\"134.544673\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"104.090341\" xlink:href=\"#m488bef92e8\" y=\"118.071946\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"92.817614\" xlink:href=\"#m488bef92e8\" y=\"126.30831\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"137.908523\" xlink:href=\"#m488bef92e8\" y=\"85.126491\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"182.999432\" xlink:href=\"#m488bef92e8\" y=\"60.417401\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"92.817614\" xlink:href=\"#m488bef92e8\" y=\"101.599219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"137.908523\" xlink:href=\"#m488bef92e8\" y=\"101.599219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"70.272159\" xlink:href=\"#m488bef92e8\" y=\"142.781037\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"126.635795\" xlink:href=\"#m488bef92e8\" y=\"126.30831\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"182.999432\" xlink:href=\"#m488bef92e8\" y=\"76.890128\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"115.363068\" xlink:href=\"#m488bef92e8\" y=\"101.599219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"115.363068\" xlink:href=\"#m488bef92e8\" y=\"134.544673\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"58.999432\" xlink:href=\"#m488bef92e8\" y=\"134.544673\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"228.090341\" xlink:href=\"#m488bef92e8\" y=\"52.181037\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"216.817614\" xlink:href=\"#m488bef92e8\" y=\"19.235582\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"182.999432\" xlink:href=\"#m488bef92e8\" y=\"60.417401\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"149.18125\" xlink:href=\"#m488bef92e8\" y=\"93.362855\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"216.817614\" xlink:href=\"#m488bef92e8\" y=\"68.653764\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"149.18125\" xlink:href=\"#m488bef92e8\" y=\"68.653764\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"182.999432\" xlink:href=\"#m488bef92e8\" y=\"101.599219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"149.18125\" xlink:href=\"#m488bef92e8\" y=\"76.890128\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"92.817614\" xlink:href=\"#m488bef92e8\" y=\"85.126491\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"149.18125\" xlink:href=\"#m488bef92e8\" y=\"109.835582\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"115.363068\" xlink:href=\"#m488bef92e8\" y=\"101.599219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"137.908523\" xlink:href=\"#m488bef92e8\" y=\"134.544673\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"137.908523\" xlink:href=\"#m488bef92e8\" y=\"101.599219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"160.453977\" xlink:href=\"#m488bef92e8\" y=\"93.362855\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"160.453977\" xlink:href=\"#m488bef92e8\" y=\"101.599219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"104.090341\" xlink:href=\"#m488bef92e8\" y=\"118.071946\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"115.363068\" xlink:href=\"#m488bef92e8\" y=\"126.30831\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"182.999432\" xlink:href=\"#m488bef92e8\" y=\"101.599219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"160.453977\" xlink:href=\"#m488bef92e8\" y=\"43.944673\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"194.272159\" xlink:href=\"#m488bef92e8\" y=\"35.70831\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"126.635795\" xlink:href=\"#m488bef92e8\" y=\"126.30831\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"137.908523\" xlink:href=\"#m488bef92e8\" y=\"118.071946\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"194.272159\" xlink:href=\"#m488bef92e8\" y=\"93.362855\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"126.635795\" xlink:href=\"#m488bef92e8\" y=\"85.126491\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"70.272159\" xlink:href=\"#m488bef92e8\" y=\"134.544673\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"149.18125\" xlink:href=\"#m488bef92e8\" y=\"101.599219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"137.908523\" xlink:href=\"#m488bef92e8\" y=\"93.362855\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"81.544886\" xlink:href=\"#m488bef92e8\" y=\"192.199219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"70.272159\" xlink:href=\"#m488bef92e8\" y=\"118.071946\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"137.908523\" xlink:href=\"#m488bef92e8\" y=\"93.362855\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"149.18125\" xlink:href=\"#m488bef92e8\" y=\"68.653764\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"115.363068\" xlink:href=\"#m488bef92e8\" y=\"134.544673\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"149.18125\" xlink:href=\"#m488bef92e8\" y=\"68.653764\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"92.817614\" xlink:href=\"#m488bef92e8\" y=\"118.071946\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"171.726705\" xlink:href=\"#m488bef92e8\" y=\"76.890128\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"137.908523\" xlink:href=\"#m488bef92e8\" y=\"109.835582\"/>\r\n    </g>\r\n   </g>\r\n   <g id=\"PathCollection_2\">\r\n    <defs>\r\n     <path d=\"M 0 3 \r\nC 0.795609 3 1.55874 2.683901 2.12132 2.12132 \r\nC 2.683901 1.55874 3 0.795609 3 0 \r\nC 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \r\nC 1.55874 -2.683901 0.795609 -3 0 -3 \r\nC -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \r\nC -2.683901 -1.55874 -3 -0.795609 -3 0 \r\nC -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \r\nC -1.55874 2.683901 -0.795609 3 0 3 \r\nz\r\n\" id=\"m2c0a81a7df\" style=\"stroke:#ff7f0e;\"/>\r\n    </defs>\r\n    <g clip-path=\"url(#pa6da11bef6)\">\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"363.363068\" xlink:href=\"#m2c0a81a7df\" y=\"118.071946\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"295.726705\" xlink:href=\"#m2c0a81a7df\" y=\"118.071946\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"352.090341\" xlink:href=\"#m2c0a81a7df\" y=\"126.30831\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"194.272159\" xlink:href=\"#m2c0a81a7df\" y=\"192.199219\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"306.999432\" xlink:href=\"#m2c0a81a7df\" y=\"151.017401\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"216.817614\" xlink:href=\"#m2c0a81a7df\" y=\"151.017401\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"284.453977\" xlink:href=\"#m2c0a81a7df\" y=\"109.835582\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"126.635795\" xlink:href=\"#m2c0a81a7df\" y=\"183.962855\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"318.272159\" xlink:href=\"#m2c0a81a7df\" y=\"142.781037\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"160.453977\" xlink:href=\"#m2c0a81a7df\" y=\"159.253764\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"137.908523\" xlink:href=\"#m2c0a81a7df\" y=\"216.90831\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"239.363068\" xlink:href=\"#m2c0a81a7df\" y=\"134.544673\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"250.635795\" xlink:href=\"#m2c0a81a7df\" y=\"200.435582\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"261.908523\" xlink:href=\"#m2c0a81a7df\" y=\"142.781037\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"205.544886\" xlink:href=\"#m2c0a81a7df\" y=\"142.781037\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"329.544886\" xlink:href=\"#m2c0a81a7df\" y=\"126.30831\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"205.544886\" xlink:href=\"#m2c0a81a7df\" y=\"134.544673\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"228.090341\" xlink:href=\"#m2c0a81a7df\" y=\"159.253764\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"273.18125\" xlink:href=\"#m2c0a81a7df\" y=\"200.435582\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"205.544886\" xlink:href=\"#m2c0a81a7df\" y=\"175.726491\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"239.363068\" xlink:href=\"#m2c0a81a7df\" y=\"118.071946\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"261.908523\" xlink:href=\"#m2c0a81a7df\" y=\"151.017401\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"284.453977\" xlink:href=\"#m2c0a81a7df\" y=\"175.726491\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"261.908523\" xlink:href=\"#m2c0a81a7df\" y=\"151.017401\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"295.726705\" xlink:href=\"#m2c0a81a7df\" y=\"142.781037\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"318.272159\" xlink:href=\"#m2c0a81a7df\" y=\"134.544673\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"340.817614\" xlink:href=\"#m2c0a81a7df\" y=\"151.017401\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"329.544886\" xlink:href=\"#m2c0a81a7df\" y=\"134.544673\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"250.635795\" xlink:href=\"#m2c0a81a7df\" y=\"142.781037\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"216.817614\" xlink:href=\"#m2c0a81a7df\" y=\"167.490128\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"194.272159\" xlink:href=\"#m2c0a81a7df\" y=\"183.962855\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"194.272159\" xlink:href=\"#m2c0a81a7df\" y=\"183.962855\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"228.090341\" xlink:href=\"#m2c0a81a7df\" y=\"159.253764\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"250.635795\" xlink:href=\"#m2c0a81a7df\" y=\"159.253764\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"182.999432\" xlink:href=\"#m2c0a81a7df\" y=\"134.544673\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"250.635795\" xlink:href=\"#m2c0a81a7df\" y=\"101.599219\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"329.544886\" xlink:href=\"#m2c0a81a7df\" y=\"126.30831\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"284.453977\" xlink:href=\"#m2c0a81a7df\" y=\"192.199219\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"205.544886\" xlink:href=\"#m2c0a81a7df\" y=\"134.544673\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"194.272159\" xlink:href=\"#m2c0a81a7df\" y=\"175.726491\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"194.272159\" xlink:href=\"#m2c0a81a7df\" y=\"167.490128\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"261.908523\" xlink:href=\"#m2c0a81a7df\" y=\"134.544673\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"228.090341\" xlink:href=\"#m2c0a81a7df\" y=\"167.490128\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"137.908523\" xlink:href=\"#m2c0a81a7df\" y=\"192.199219\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"205.544886\" xlink:href=\"#m2c0a81a7df\" y=\"159.253764\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"216.817614\" xlink:href=\"#m2c0a81a7df\" y=\"134.544673\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"216.817614\" xlink:href=\"#m2c0a81a7df\" y=\"142.781037\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"273.18125\" xlink:href=\"#m2c0a81a7df\" y=\"142.781037\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"149.18125\" xlink:href=\"#m2c0a81a7df\" y=\"175.726491\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"216.817614\" xlink:href=\"#m2c0a81a7df\" y=\"151.017401\"/>\r\n    </g>\r\n   </g>\r\n   <g id=\"matplotlib.axis_1\">\r\n    <g id=\"xtick_1\">\r\n     <g id=\"line2d_1\">\r\n      <defs>\r\n       <path d=\"M 0 0 \r\nL 0 3.5 \r\n\" id=\"m0c43572c7e\" style=\"stroke:#000000;stroke-width:0.8;\"/>\r\n      </defs>\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"81.544886\" xlink:href=\"#m0c43572c7e\" y=\"226.791946\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_1\">\r\n      <!-- 4.5 -->\r\n      <defs>\r\n       <path d=\"M 37.796875 64.3125 \r\nL 12.890625 25.390625 \r\nL 37.796875 25.390625 \r\nz\r\nM 35.203125 72.90625 \r\nL 47.609375 72.90625 \r\nL 47.609375 25.390625 \r\nL 58.015625 25.390625 \r\nL 58.015625 17.1875 \r\nL 47.609375 17.1875 \r\nL 47.609375 0 \r\nL 37.796875 0 \r\nL 37.796875 17.1875 \r\nL 4.890625 17.1875 \r\nL 4.890625 26.703125 \r\nz\r\n\" id=\"DejaVuSans-52\"/>\r\n       <path d=\"M 10.6875 12.40625 \r\nL 21 12.40625 \r\nL 21 0 \r\nL 10.6875 0 \r\nz\r\n\" id=\"DejaVuSans-46\"/>\r\n       <path d=\"M 10.796875 72.90625 \r\nL 49.515625 72.90625 \r\nL 49.515625 64.59375 \r\nL 19.828125 64.59375 \r\nL 19.828125 46.734375 \r\nQ 21.96875 47.46875 24.109375 47.828125 \r\nQ 26.265625 48.1875 28.421875 48.1875 \r\nQ 40.625 48.1875 47.75 41.5 \r\nQ 54.890625 34.8125 54.890625 23.390625 \r\nQ 54.890625 11.625 47.5625 5.09375 \r\nQ 40.234375 -1.421875 26.90625 -1.421875 \r\nQ 22.3125 -1.421875 17.546875 -0.640625 \r\nQ 12.796875 0.140625 7.71875 1.703125 \r\nL 7.71875 11.625 \r\nQ 12.109375 9.234375 16.796875 8.0625 \r\nQ 21.484375 6.890625 26.703125 6.890625 \r\nQ 35.15625 6.890625 40.078125 11.328125 \r\nQ 45.015625 15.765625 45.015625 23.390625 \r\nQ 45.015625 31 40.078125 35.4375 \r\nQ 35.15625 39.890625 26.703125 39.890625 \r\nQ 22.75 39.890625 18.8125 39.015625 \r\nQ 14.890625 38.140625 10.796875 36.28125 \r\nz\r\n\" id=\"DejaVuSans-53\"/>\r\n      </defs>\r\n      <g transform=\"translate(73.593324 241.390384)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-52\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"xtick_2\">\r\n     <g id=\"line2d_2\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"137.908523\" xlink:href=\"#m0c43572c7e\" y=\"226.791946\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_2\">\r\n      <!-- 5.0 -->\r\n      <defs>\r\n       <path d=\"M 31.78125 66.40625 \r\nQ 24.171875 66.40625 20.328125 58.90625 \r\nQ 16.5 51.421875 16.5 36.375 \r\nQ 16.5 21.390625 20.328125 13.890625 \r\nQ 24.171875 6.390625 31.78125 6.390625 \r\nQ 39.453125 6.390625 43.28125 13.890625 \r\nQ 47.125 21.390625 47.125 36.375 \r\nQ 47.125 51.421875 43.28125 58.90625 \r\nQ 39.453125 66.40625 31.78125 66.40625 \r\nz\r\nM 31.78125 74.21875 \r\nQ 44.046875 74.21875 50.515625 64.515625 \r\nQ 56.984375 54.828125 56.984375 36.375 \r\nQ 56.984375 17.96875 50.515625 8.265625 \r\nQ 44.046875 -1.421875 31.78125 -1.421875 \r\nQ 19.53125 -1.421875 13.0625 8.265625 \r\nQ 6.59375 17.96875 6.59375 36.375 \r\nQ 6.59375 54.828125 13.0625 64.515625 \r\nQ 19.53125 74.21875 31.78125 74.21875 \r\nz\r\n\" id=\"DejaVuSans-48\"/>\r\n      </defs>\r\n      <g transform=\"translate(129.95696 241.390384)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-53\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"xtick_3\">\r\n     <g id=\"line2d_3\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"194.272159\" xlink:href=\"#m0c43572c7e\" y=\"226.791946\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_3\">\r\n      <!-- 5.5 -->\r\n      <g transform=\"translate(186.320597 241.390384)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-53\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"xtick_4\">\r\n     <g id=\"line2d_4\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"250.635795\" xlink:href=\"#m0c43572c7e\" y=\"226.791946\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_4\">\r\n      <!-- 6.0 -->\r\n      <defs>\r\n       <path d=\"M 33.015625 40.375 \r\nQ 26.375 40.375 22.484375 35.828125 \r\nQ 18.609375 31.296875 18.609375 23.390625 \r\nQ 18.609375 15.53125 22.484375 10.953125 \r\nQ 26.375 6.390625 33.015625 6.390625 \r\nQ 39.65625 6.390625 43.53125 10.953125 \r\nQ 47.40625 15.53125 47.40625 23.390625 \r\nQ 47.40625 31.296875 43.53125 35.828125 \r\nQ 39.65625 40.375 33.015625 40.375 \r\nz\r\nM 52.59375 71.296875 \r\nL 52.59375 62.3125 \r\nQ 48.875 64.0625 45.09375 64.984375 \r\nQ 41.3125 65.921875 37.59375 65.921875 \r\nQ 27.828125 65.921875 22.671875 59.328125 \r\nQ 17.53125 52.734375 16.796875 39.40625 \r\nQ 19.671875 43.65625 24.015625 45.921875 \r\nQ 28.375 48.1875 33.59375 48.1875 \r\nQ 44.578125 48.1875 50.953125 41.515625 \r\nQ 57.328125 34.859375 57.328125 23.390625 \r\nQ 57.328125 12.15625 50.6875 5.359375 \r\nQ 44.046875 -1.421875 33.015625 -1.421875 \r\nQ 20.359375 -1.421875 13.671875 8.265625 \r\nQ 6.984375 17.96875 6.984375 36.375 \r\nQ 6.984375 53.65625 15.1875 63.9375 \r\nQ 23.390625 74.21875 37.203125 74.21875 \r\nQ 40.921875 74.21875 44.703125 73.484375 \r\nQ 48.484375 72.75 52.59375 71.296875 \r\nz\r\n\" id=\"DejaVuSans-54\"/>\r\n      </defs>\r\n      <g transform=\"translate(242.684233 241.390384)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-54\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"xtick_5\">\r\n     <g id=\"line2d_5\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"306.999432\" xlink:href=\"#m0c43572c7e\" y=\"226.791946\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_5\">\r\n      <!-- 6.5 -->\r\n      <g transform=\"translate(299.047869 241.390384)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-54\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"xtick_6\">\r\n     <g id=\"line2d_6\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"363.363068\" xlink:href=\"#m0c43572c7e\" y=\"226.791946\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_6\">\r\n      <!-- 7.0 -->\r\n      <defs>\r\n       <path d=\"M 8.203125 72.90625 \r\nL 55.078125 72.90625 \r\nL 55.078125 68.703125 \r\nL 28.609375 0 \r\nL 18.3125 0 \r\nL 43.21875 64.59375 \r\nL 8.203125 64.59375 \r\nz\r\n\" id=\"DejaVuSans-55\"/>\r\n      </defs>\r\n      <g transform=\"translate(355.411506 241.390384)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-55\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"text_7\">\r\n     <!-- sepal length -->\r\n     <defs>\r\n      <path d=\"M 44.28125 53.078125 \r\nL 44.28125 44.578125 \r\nQ 40.484375 46.53125 36.375 47.5 \r\nQ 32.28125 48.484375 27.875 48.484375 \r\nQ 21.1875 48.484375 17.84375 46.4375 \r\nQ 14.5 44.390625 14.5 40.28125 \r\nQ 14.5 37.15625 16.890625 35.375 \r\nQ 19.28125 33.59375 26.515625 31.984375 \r\nL 29.59375 31.296875 \r\nQ 39.15625 29.25 43.1875 25.515625 \r\nQ 47.21875 21.78125 47.21875 15.09375 \r\nQ 47.21875 7.46875 41.1875 3.015625 \r\nQ 35.15625 -1.421875 24.609375 -1.421875 \r\nQ 20.21875 -1.421875 15.453125 -0.5625 \r\nQ 10.6875 0.296875 5.421875 2 \r\nL 5.421875 11.28125 \r\nQ 10.40625 8.6875 15.234375 7.390625 \r\nQ 20.0625 6.109375 24.8125 6.109375 \r\nQ 31.15625 6.109375 34.5625 8.28125 \r\nQ 37.984375 10.453125 37.984375 14.40625 \r\nQ 37.984375 18.0625 35.515625 20.015625 \r\nQ 33.0625 21.96875 24.703125 23.78125 \r\nL 21.578125 24.515625 \r\nQ 13.234375 26.265625 9.515625 29.90625 \r\nQ 5.8125 33.546875 5.8125 39.890625 \r\nQ 5.8125 47.609375 11.28125 51.796875 \r\nQ 16.75 56 26.8125 56 \r\nQ 31.78125 56 36.171875 55.265625 \r\nQ 40.578125 54.546875 44.28125 53.078125 \r\nz\r\n\" id=\"DejaVuSans-115\"/>\r\n      <path d=\"M 56.203125 29.59375 \r\nL 56.203125 25.203125 \r\nL 14.890625 25.203125 \r\nQ 15.484375 15.921875 20.484375 11.0625 \r\nQ 25.484375 6.203125 34.421875 6.203125 \r\nQ 39.59375 6.203125 44.453125 7.46875 \r\nQ 49.3125 8.734375 54.109375 11.28125 \r\nL 54.109375 2.78125 \r\nQ 49.265625 0.734375 44.1875 -0.34375 \r\nQ 39.109375 -1.421875 33.890625 -1.421875 \r\nQ 20.796875 -1.421875 13.15625 6.1875 \r\nQ 5.515625 13.8125 5.515625 26.8125 \r\nQ 5.515625 40.234375 12.765625 48.109375 \r\nQ 20.015625 56 32.328125 56 \r\nQ 43.359375 56 49.78125 48.890625 \r\nQ 56.203125 41.796875 56.203125 29.59375 \r\nz\r\nM 47.21875 32.234375 \r\nQ 47.125 39.59375 43.09375 43.984375 \r\nQ 39.0625 48.390625 32.421875 48.390625 \r\nQ 24.90625 48.390625 20.390625 44.140625 \r\nQ 15.875 39.890625 15.1875 32.171875 \r\nz\r\n\" id=\"DejaVuSans-101\"/>\r\n      <path d=\"M 18.109375 8.203125 \r\nL 18.109375 -20.796875 \r\nL 9.078125 -20.796875 \r\nL 9.078125 54.6875 \r\nL 18.109375 54.6875 \r\nL 18.109375 46.390625 \r\nQ 20.953125 51.265625 25.265625 53.625 \r\nQ 29.59375 56 35.59375 56 \r\nQ 45.5625 56 51.78125 48.09375 \r\nQ 58.015625 40.1875 58.015625 27.296875 \r\nQ 58.015625 14.40625 51.78125 6.484375 \r\nQ 45.5625 -1.421875 35.59375 -1.421875 \r\nQ 29.59375 -1.421875 25.265625 0.953125 \r\nQ 20.953125 3.328125 18.109375 8.203125 \r\nz\r\nM 48.6875 27.296875 \r\nQ 48.6875 37.203125 44.609375 42.84375 \r\nQ 40.53125 48.484375 33.40625 48.484375 \r\nQ 26.265625 48.484375 22.1875 42.84375 \r\nQ 18.109375 37.203125 18.109375 27.296875 \r\nQ 18.109375 17.390625 22.1875 11.75 \r\nQ 26.265625 6.109375 33.40625 6.109375 \r\nQ 40.53125 6.109375 44.609375 11.75 \r\nQ 48.6875 17.390625 48.6875 27.296875 \r\nz\r\n\" id=\"DejaVuSans-112\"/>\r\n      <path d=\"M 34.28125 27.484375 \r\nQ 23.390625 27.484375 19.1875 25 \r\nQ 14.984375 22.515625 14.984375 16.5 \r\nQ 14.984375 11.71875 18.140625 8.90625 \r\nQ 21.296875 6.109375 26.703125 6.109375 \r\nQ 34.1875 6.109375 38.703125 11.40625 \r\nQ 43.21875 16.703125 43.21875 25.484375 \r\nL 43.21875 27.484375 \r\nz\r\nM 52.203125 31.203125 \r\nL 52.203125 0 \r\nL 43.21875 0 \r\nL 43.21875 8.296875 \r\nQ 40.140625 3.328125 35.546875 0.953125 \r\nQ 30.953125 -1.421875 24.3125 -1.421875 \r\nQ 15.921875 -1.421875 10.953125 3.296875 \r\nQ 6 8.015625 6 15.921875 \r\nQ 6 25.140625 12.171875 29.828125 \r\nQ 18.359375 34.515625 30.609375 34.515625 \r\nL 43.21875 34.515625 \r\nL 43.21875 35.40625 \r\nQ 43.21875 41.609375 39.140625 45 \r\nQ 35.0625 48.390625 27.6875 48.390625 \r\nQ 23 48.390625 18.546875 47.265625 \r\nQ 14.109375 46.140625 10.015625 43.890625 \r\nL 10.015625 52.203125 \r\nQ 14.9375 54.109375 19.578125 55.046875 \r\nQ 24.21875 56 28.609375 56 \r\nQ 40.484375 56 46.34375 49.84375 \r\nQ 52.203125 43.703125 52.203125 31.203125 \r\nz\r\n\" id=\"DejaVuSans-97\"/>\r\n      <path d=\"M 9.421875 75.984375 \r\nL 18.40625 75.984375 \r\nL 18.40625 0 \r\nL 9.421875 0 \r\nz\r\n\" id=\"DejaVuSans-108\"/>\r\n      <path id=\"DejaVuSans-32\"/>\r\n      <path d=\"M 54.890625 33.015625 \r\nL 54.890625 0 \r\nL 45.90625 0 \r\nL 45.90625 32.71875 \r\nQ 45.90625 40.484375 42.875 44.328125 \r\nQ 39.84375 48.1875 33.796875 48.1875 \r\nQ 26.515625 48.1875 22.3125 43.546875 \r\nQ 18.109375 38.921875 18.109375 30.90625 \r\nL 18.109375 0 \r\nL 9.078125 0 \r\nL 9.078125 54.6875 \r\nL 18.109375 54.6875 \r\nL 18.109375 46.1875 \r\nQ 21.34375 51.125 25.703125 53.5625 \r\nQ 30.078125 56 35.796875 56 \r\nQ 45.21875 56 50.046875 50.171875 \r\nQ 54.890625 44.34375 54.890625 33.015625 \r\nz\r\n\" id=\"DejaVuSans-110\"/>\r\n      <path d=\"M 45.40625 27.984375 \r\nQ 45.40625 37.75 41.375 43.109375 \r\nQ 37.359375 48.484375 30.078125 48.484375 \r\nQ 22.859375 48.484375 18.828125 43.109375 \r\nQ 14.796875 37.75 14.796875 27.984375 \r\nQ 14.796875 18.265625 18.828125 12.890625 \r\nQ 22.859375 7.515625 30.078125 7.515625 \r\nQ 37.359375 7.515625 41.375 12.890625 \r\nQ 45.40625 18.265625 45.40625 27.984375 \r\nz\r\nM 54.390625 6.78125 \r\nQ 54.390625 -7.171875 48.1875 -13.984375 \r\nQ 42 -20.796875 29.203125 -20.796875 \r\nQ 24.46875 -20.796875 20.265625 -20.09375 \r\nQ 16.0625 -19.390625 12.109375 -17.921875 \r\nL 12.109375 -9.1875 \r\nQ 16.0625 -11.328125 19.921875 -12.34375 \r\nQ 23.78125 -13.375 27.78125 -13.375 \r\nQ 36.625 -13.375 41.015625 -8.765625 \r\nQ 45.40625 -4.15625 45.40625 5.171875 \r\nL 45.40625 9.625 \r\nQ 42.625 4.78125 38.28125 2.390625 \r\nQ 33.9375 0 27.875 0 \r\nQ 17.828125 0 11.671875 7.65625 \r\nQ 5.515625 15.328125 5.515625 27.984375 \r\nQ 5.515625 40.671875 11.671875 48.328125 \r\nQ 17.828125 56 27.875 56 \r\nQ 33.9375 56 38.28125 53.609375 \r\nQ 42.625 51.21875 45.40625 46.390625 \r\nL 45.40625 54.6875 \r\nL 54.390625 54.6875 \r\nz\r\n\" id=\"DejaVuSans-103\"/>\r\n      <path d=\"M 18.3125 70.21875 \r\nL 18.3125 54.6875 \r\nL 36.8125 54.6875 \r\nL 36.8125 47.703125 \r\nL 18.3125 47.703125 \r\nL 18.3125 18.015625 \r\nQ 18.3125 11.328125 20.140625 9.421875 \r\nQ 21.96875 7.515625 27.59375 7.515625 \r\nL 36.8125 7.515625 \r\nL 36.8125 0 \r\nL 27.59375 0 \r\nQ 17.1875 0 13.234375 3.875 \r\nQ 9.28125 7.765625 9.28125 18.015625 \r\nL 9.28125 47.703125 \r\nL 2.6875 47.703125 \r\nL 2.6875 54.6875 \r\nL 9.28125 54.6875 \r\nL 9.28125 70.21875 \r\nz\r\n\" id=\"DejaVuSans-116\"/>\r\n      <path d=\"M 54.890625 33.015625 \r\nL 54.890625 0 \r\nL 45.90625 0 \r\nL 45.90625 32.71875 \r\nQ 45.90625 40.484375 42.875 44.328125 \r\nQ 39.84375 48.1875 33.796875 48.1875 \r\nQ 26.515625 48.1875 22.3125 43.546875 \r\nQ 18.109375 38.921875 18.109375 30.90625 \r\nL 18.109375 0 \r\nL 9.078125 0 \r\nL 9.078125 75.984375 \r\nL 18.109375 75.984375 \r\nL 18.109375 46.1875 \r\nQ 21.34375 51.125 25.703125 53.5625 \r\nQ 30.078125 56 35.796875 56 \r\nQ 45.21875 56 50.046875 50.171875 \r\nQ 54.890625 44.34375 54.890625 33.015625 \r\nz\r\n\" id=\"DejaVuSans-104\"/>\r\n     </defs>\r\n     <g transform=\"translate(180.346094 255.068509)scale(0.1 -0.1)\">\r\n      <use xlink:href=\"#DejaVuSans-115\"/>\r\n      <use x=\"52.099609\" xlink:href=\"#DejaVuSans-101\"/>\r\n      <use x=\"113.623047\" xlink:href=\"#DejaVuSans-112\"/>\r\n      <use x=\"177.099609\" xlink:href=\"#DejaVuSans-97\"/>\r\n      <use x=\"238.378906\" xlink:href=\"#DejaVuSans-108\"/>\r\n      <use x=\"266.162109\" xlink:href=\"#DejaVuSans-32\"/>\r\n      <use x=\"297.949219\" xlink:href=\"#DejaVuSans-108\"/>\r\n      <use x=\"325.732422\" xlink:href=\"#DejaVuSans-101\"/>\r\n      <use x=\"387.255859\" xlink:href=\"#DejaVuSans-110\"/>\r\n      <use x=\"450.634766\" xlink:href=\"#DejaVuSans-103\"/>\r\n      <use x=\"514.111328\" xlink:href=\"#DejaVuSans-116\"/>\r\n      <use x=\"553.320312\" xlink:href=\"#DejaVuSans-104\"/>\r\n     </g>\r\n    </g>\r\n   </g>\r\n   <g id=\"matplotlib.axis_2\">\r\n    <g id=\"ytick_1\">\r\n     <g id=\"line2d_7\">\r\n      <defs>\r\n       <path d=\"M 0 0 \r\nL -3.5 0 \r\n\" id=\"m675889ff83\" style=\"stroke:#000000;stroke-width:0.8;\"/>\r\n      </defs>\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#m675889ff83\" y=\"216.90831\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_8\">\r\n      <!-- 2.0 -->\r\n      <defs>\r\n       <path d=\"M 19.1875 8.296875 \r\nL 53.609375 8.296875 \r\nL 53.609375 0 \r\nL 7.328125 0 \r\nL 7.328125 8.296875 \r\nQ 12.9375 14.109375 22.625 23.890625 \r\nQ 32.328125 33.6875 34.8125 36.53125 \r\nQ 39.546875 41.84375 41.421875 45.53125 \r\nQ 43.3125 49.21875 43.3125 52.78125 \r\nQ 43.3125 58.59375 39.234375 62.25 \r\nQ 35.15625 65.921875 28.609375 65.921875 \r\nQ 23.96875 65.921875 18.8125 64.3125 \r\nQ 13.671875 62.703125 7.8125 59.421875 \r\nL 7.8125 69.390625 \r\nQ 13.765625 71.78125 18.9375 73 \r\nQ 24.125 74.21875 28.421875 74.21875 \r\nQ 39.75 74.21875 46.484375 68.546875 \r\nQ 53.21875 62.890625 53.21875 53.421875 \r\nQ 53.21875 48.921875 51.53125 44.890625 \r\nQ 49.859375 40.875 45.40625 35.40625 \r\nQ 44.1875 33.984375 37.640625 27.21875 \r\nQ 31.109375 20.453125 19.1875 8.296875 \r\nz\r\n\" id=\"DejaVuSans-50\"/>\r\n      </defs>\r\n      <g transform=\"translate(20.878125 220.707528)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-50\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"ytick_2\">\r\n     <g id=\"line2d_8\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#m675889ff83\" y=\"175.726491\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_9\">\r\n      <!-- 2.5 -->\r\n      <g transform=\"translate(20.878125 179.52571)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-50\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"ytick_3\">\r\n     <g id=\"line2d_9\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#m675889ff83\" y=\"134.544673\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_10\">\r\n      <!-- 3.0 -->\r\n      <defs>\r\n       <path d=\"M 40.578125 39.3125 \r\nQ 47.65625 37.796875 51.625 33 \r\nQ 55.609375 28.21875 55.609375 21.1875 \r\nQ 55.609375 10.40625 48.1875 4.484375 \r\nQ 40.765625 -1.421875 27.09375 -1.421875 \r\nQ 22.515625 -1.421875 17.65625 -0.515625 \r\nQ 12.796875 0.390625 7.625 2.203125 \r\nL 7.625 11.71875 \r\nQ 11.71875 9.328125 16.59375 8.109375 \r\nQ 21.484375 6.890625 26.8125 6.890625 \r\nQ 36.078125 6.890625 40.9375 10.546875 \r\nQ 45.796875 14.203125 45.796875 21.1875 \r\nQ 45.796875 27.640625 41.28125 31.265625 \r\nQ 36.765625 34.90625 28.71875 34.90625 \r\nL 20.21875 34.90625 \r\nL 20.21875 43.015625 \r\nL 29.109375 43.015625 \r\nQ 36.375 43.015625 40.234375 45.921875 \r\nQ 44.09375 48.828125 44.09375 54.296875 \r\nQ 44.09375 59.90625 40.109375 62.90625 \r\nQ 36.140625 65.921875 28.71875 65.921875 \r\nQ 24.65625 65.921875 20.015625 65.03125 \r\nQ 15.375 64.15625 9.8125 62.3125 \r\nL 9.8125 71.09375 \r\nQ 15.4375 72.65625 20.34375 73.4375 \r\nQ 25.25 74.21875 29.59375 74.21875 \r\nQ 40.828125 74.21875 47.359375 69.109375 \r\nQ 53.90625 64.015625 53.90625 55.328125 \r\nQ 53.90625 49.265625 50.4375 45.09375 \r\nQ 46.96875 40.921875 40.578125 39.3125 \r\nz\r\n\" id=\"DejaVuSans-51\"/>\r\n      </defs>\r\n      <g transform=\"translate(20.878125 138.343892)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-51\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"ytick_4\">\r\n     <g id=\"line2d_10\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#m675889ff83\" y=\"93.362855\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_11\">\r\n      <!-- 3.5 -->\r\n      <g transform=\"translate(20.878125 97.162074)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-51\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"ytick_5\">\r\n     <g id=\"line2d_11\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#m675889ff83\" y=\"52.181037\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_12\">\r\n      <!-- 4.0 -->\r\n      <g transform=\"translate(20.878125 55.980256)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-52\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"ytick_6\">\r\n     <g id=\"line2d_12\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#m675889ff83\" y=\"10.999219\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_13\">\r\n      <!-- 4.5 -->\r\n      <g transform=\"translate(20.878125 14.798437)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-52\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"text_14\">\r\n     <!-- sepal width -->\r\n     <defs>\r\n      <path d=\"M 4.203125 54.6875 \r\nL 13.1875 54.6875 \r\nL 24.421875 12.015625 \r\nL 35.59375 54.6875 \r\nL 46.1875 54.6875 \r\nL 57.421875 12.015625 \r\nL 68.609375 54.6875 \r\nL 77.59375 54.6875 \r\nL 63.28125 0 \r\nL 52.6875 0 \r\nL 40.921875 44.828125 \r\nL 29.109375 0 \r\nL 18.5 0 \r\nz\r\n\" id=\"DejaVuSans-119\"/>\r\n      <path d=\"M 9.421875 54.6875 \r\nL 18.40625 54.6875 \r\nL 18.40625 0 \r\nL 9.421875 0 \r\nz\r\nM 9.421875 75.984375 \r\nL 18.40625 75.984375 \r\nL 18.40625 64.59375 \r\nL 9.421875 64.59375 \r\nz\r\n\" id=\"DejaVuSans-105\"/>\r\n      <path d=\"M 45.40625 46.390625 \r\nL 45.40625 75.984375 \r\nL 54.390625 75.984375 \r\nL 54.390625 0 \r\nL 45.40625 0 \r\nL 45.40625 8.203125 \r\nQ 42.578125 3.328125 38.25 0.953125 \r\nQ 33.9375 -1.421875 27.875 -1.421875 \r\nQ 17.96875 -1.421875 11.734375 6.484375 \r\nQ 5.515625 14.40625 5.515625 27.296875 \r\nQ 5.515625 40.1875 11.734375 48.09375 \r\nQ 17.96875 56 27.875 56 \r\nQ 33.9375 56 38.25 53.625 \r\nQ 42.578125 51.265625 45.40625 46.390625 \r\nz\r\nM 14.796875 27.296875 \r\nQ 14.796875 17.390625 18.875 11.75 \r\nQ 22.953125 6.109375 30.078125 6.109375 \r\nQ 37.203125 6.109375 41.296875 11.75 \r\nQ 45.40625 17.390625 45.40625 27.296875 \r\nQ 45.40625 37.203125 41.296875 42.84375 \r\nQ 37.203125 48.484375 30.078125 48.484375 \r\nQ 22.953125 48.484375 18.875 42.84375 \r\nQ 14.796875 37.203125 14.796875 27.296875 \r\nz\r\n\" id=\"DejaVuSans-100\"/>\r\n     </defs>\r\n     <g transform=\"translate(14.798438 146.750852)rotate(-90)scale(0.1 -0.1)\">\r\n      <use xlink:href=\"#DejaVuSans-115\"/>\r\n      <use x=\"52.099609\" xlink:href=\"#DejaVuSans-101\"/>\r\n      <use x=\"113.623047\" xlink:href=\"#DejaVuSans-112\"/>\r\n      <use x=\"177.099609\" xlink:href=\"#DejaVuSans-97\"/>\r\n      <use x=\"238.378906\" xlink:href=\"#DejaVuSans-108\"/>\r\n      <use x=\"266.162109\" xlink:href=\"#DejaVuSans-32\"/>\r\n      <use x=\"297.949219\" xlink:href=\"#DejaVuSans-119\"/>\r\n      <use x=\"379.736328\" xlink:href=\"#DejaVuSans-105\"/>\r\n      <use x=\"407.519531\" xlink:href=\"#DejaVuSans-100\"/>\r\n      <use x=\"470.996094\" xlink:href=\"#DejaVuSans-116\"/>\r\n      <use x=\"510.205078\" xlink:href=\"#DejaVuSans-104\"/>\r\n     </g>\r\n    </g>\r\n   </g>\r\n   <g id=\"patch_3\">\r\n    <path d=\"M 43.78125 226.791946 \r\nL 43.78125 9.351946 \r\n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n   </g>\r\n   <g id=\"patch_4\">\r\n    <path d=\"M 378.58125 226.791946 \r\nL 378.58125 9.351946 \r\n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n   </g>\r\n   <g id=\"patch_5\">\r\n    <path d=\"M 43.78125 226.791946 \r\nL 378.58125 226.791946 \r\n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n   </g>\r\n   <g id=\"patch_6\">\r\n    <path d=\"M 43.78125 9.351946 \r\nL 378.58125 9.351946 \r\n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n   </g>\r\n   <g id=\"legend_1\">\r\n    <g id=\"patch_7\">\r\n     <path d=\"M 333.21875 46.708196 \r\nL 371.58125 46.708196 \r\nQ 373.58125 46.708196 373.58125 44.708196 \r\nL 373.58125 16.351946 \r\nQ 373.58125 14.351946 371.58125 14.351946 \r\nL 333.21875 14.351946 \r\nQ 331.21875 14.351946 331.21875 16.351946 \r\nL 331.21875 44.708196 \r\nQ 331.21875 46.708196 333.21875 46.708196 \r\nz\r\n\" style=\"fill:#ffffff;opacity:0.8;stroke:#cccccc;stroke-linejoin:miter;\"/>\r\n    </g>\r\n    <g id=\"PathCollection_3\">\r\n     <g>\r\n      <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"345.21875\" xlink:href=\"#m488bef92e8\" y=\"23.325384\"/>\r\n     </g>\r\n    </g>\r\n    <g id=\"text_15\">\r\n     <!-- 0 -->\r\n     <g transform=\"translate(363.21875 25.950384)scale(0.1 -0.1)\">\r\n      <use xlink:href=\"#DejaVuSans-48\"/>\r\n     </g>\r\n    </g>\r\n    <g id=\"PathCollection_4\">\r\n     <g>\r\n      <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"345.21875\" xlink:href=\"#m2c0a81a7df\" y=\"38.003509\"/>\r\n     </g>\r\n    </g>\r\n    <g id=\"text_16\">\r\n     <!-- 1 -->\r\n     <defs>\r\n      <path d=\"M 12.40625 8.296875 \r\nL 28.515625 8.296875 \r\nL 28.515625 63.921875 \r\nL 10.984375 60.40625 \r\nL 10.984375 69.390625 \r\nL 28.421875 72.90625 \r\nL 38.28125 72.90625 \r\nL 38.28125 8.296875 \r\nL 54.390625 8.296875 \r\nL 54.390625 0 \r\nL 12.40625 0 \r\nz\r\n\" id=\"DejaVuSans-49\"/>\r\n     </defs>\r\n     <g transform=\"translate(363.21875 40.628509)scale(0.1 -0.1)\">\r\n      <use xlink:href=\"#DejaVuSans-49\"/>\r\n     </g>\r\n    </g>\r\n   </g>\r\n  </g>\r\n </g>\r\n <defs>\r\n  <clipPath id=\"pa6da11bef6\">\r\n   <rect height=\"217.44\" width=\"334.8\" x=\"43.78125\" y=\"9.351946\"/>\r\n  </clipPath>\r\n </defs>\r\n</svg>\r\n",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEJCAYAAACZjSCSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAffElEQVR4nO3df5RddXnv8fenYTRRgVwhVsgkRoWbqwQkMIIYiwq2aIiQIuXCqnpRV1O9VnFRcRVr1Ytt0WLVIveCKFYt3tBoMSjyQy+IopbQ/IBEEiNYtPnB1TTeBJAgIX3uH3tPMjmcmTl75nzP2T8+r7VmzZx99vnOs/eGebL3fp79VURgZmbN9Vv9DsDMzPrLicDMrOGcCMzMGs6JwMys4ZwIzMwazonAzKzhkicCSVMkrZF0Y5v3XiVpp6R78q8Ppo7HzMz2d0APfscFwAbgoFHevzMiFvUgDjMzayNpIpA0CJwO/BVwYTfGPPTQQ2POnDndGMrMrDFWrVr17xExo917qc8IPgW8DzhwjHVOknQvsBV4b0TcN9aAc+bMYeXKlV0M0cys/iT9fLT3kt0jkLQI+GVErBpjtdXA8yLiJcCngeWjjLVE0kpJK7dt25YgWjOz5kp5s3gBcIaknwHXAadIunbkChHxcEQ8mv98EzAg6dDWgSLi6ogYioihGTPantmYmdkEJUsEEXFxRAxGxBzgXOD2iHjjyHUkPVeS8p9PyOPZniomMzN7ql5UDe1H0tsBIuIq4GzgHZKeBHYB54Yfh2pmJbV79242b97M448/3u9QRjV16lQGBwcZGBjo+DOq2t/doaGh8M1iM+uHBx98kAMPPJBDDjmE/GJGqUQE27dv55FHHuH5z3/+fu9JWhURQ+0+1/MzArOmWL5mC5fdupGtO3Zx+PRpXHTaXBbPn9nvsGwSHn/8cebMmVPKJAAgiUMOOYSiRTVOBGYJLF+zhYuvX8eu3XsA2LJjFxdfvw7AyaDiypoEhk0kPj9ryCyBy27duDcJDNu1ew+X3bqxTxGZjc6JwCyBrTt2FVpu1qlbbrmFuXPncsQRR/DRj360K2M6EZglcPj0aYWWm3Viz549vPOd7+Tmm29m/fr1LF26lPXr1096XCcCswQuOm0u0wam7Lds2sAULjptbp8isn5YvmYLCz56O8//s2+y4KO3s3zNlkmNd/fdd3PEEUfwghe8gKc97Wmce+653HDDDZOO04nALIHF82dy6VlHM3P6NATMnD6NS8862jeKG2S4YGDLjl0E+woGJpMMtmzZwqxZs/a+HhwcZMuWySUXcNWQWTKL58/0H/4GG6tgYKL/XbTr++pGFZPPCMzMEkhRMDA4OMimTZv2vt68eTOHH374hMcb5kRgZpZAioKBl770pdx///08+OCDPPHEE1x33XWcccYZEx5vmBOBmVkCKQoGDjjgAK644gpOO+00XvSiF3HOOedw1FFHTTZU3yMwM0th+D5Atx8zsnDhQhYuXNiNEPdyIjAzS6QqBQO+NGRm1nBOBGZmDedEYGbWcE4EZmYN55vF1nieQMaazmcE1mgpngdjltJb3/pWnvOc5zBv3ryujelEYI3mCWSsas4//3xuueWWro7pRGCN5glkLKm1y+CT8+DD07Pva5dNesiTTz6ZZz/72V0Ibh8nAms0TyBjyaxdBt94N+zcBET2/Rvv7koy6DYnAms0TyBjydx2CexuObPcvStbXjKuGrJGS/U8GDN2bi62vI+cCKzxqvI8GKuYgwfzy0JtlpeMLw1Z33R7PlezUjn1gzDQcq9pYFq2fBLOO+88TjrpJDZu3Mjg4CDXXHPNpMYDnxFYnwzX7w+Xbg7X7wP+17nVwzHnZN9vuyS7HHTwYJYEhpdP0NKlS7sQ3P6cCKwvUsznalY6x5wz6T/8veBLQ9YXrt83Kw8nAusL1+9bVUVEv0MY00TicyKwvnD9vlXR1KlT2b59e2mTQUSwfft2pk6dWuhzvkdgfeH6fauiwcFBNm/ezLZt2/odyqimTp3K4GCxElWlzmySpgArgS0RsajlPQF/BywEHgPOj4jVY403NDQUK1euTBWumVktSVoVEUPt3uvFGcEFwAbgoDbvvQ44Mv86Ebgy/27WOJ4Xwfol6T0CSYPA6cDnRlnlTOBLkbkLmC7psJQxmZWR50Wwfkp9s/hTwPuA/xjl/ZnAyB7szfkys0bxvAjWT8kSgaRFwC8jYtVYq7VZ9pSbFpKWSFopaWWZb9KYTZT7KqyfUp4RLADOkPQz4DrgFEnXtqyzGZg14vUgsLV1oIi4OiKGImJoxowZqeI16xv3VVg/JUsEEXFxRAxGxBzgXOD2iHhjy2pfB96szMuAnRHxUKqYzMrKfRXWTz3vI5D0doCIuAq4iax09AGy8tG39DoeszJwX4X1U/I+gm5zH4GZWXH97iMw66kPLF/H0hWb2BPBFInzTpzFXy4+ut9hmZWWE4HVygeWr+Pau/5t7+s9EXtfOxmYteeHzlmtLF3RZmrAMZabmROB1cyeUe55jbbczJwIrGamqF2P4ujLzcyJwGrmvBNnFVpuZr5ZbDUzfEPYVUNmnXMfgZlZA4zVR+BLQ2ZmDedLQ9ZVf/jZf+YHP/3V3tcLXvhsvvxHJ/Uxov7xRDNWFT4jsK5pTQIAP/jpr/jDz/5znyLqH080Y1XiRGBd05oExlteZ55oxqrEicAsAU80Y1XiRGCWgCeasSpxIrCuWfDCZxdaXmeeaMaqxInAuubLf3TSU/7oN7VqaPH8mVx61tHMnD4NATOnT+PSs4521ZCVkhvKzMwawBPTWM+kqp0vMq7r982KcSKwrhmunR8umxyunQcm9Ye4yLipYjCrM98jsK5JVTtfZFzX75sV50RgXZOqdr7IuK7fNyvOicC6JlXtfJFxXb9vVpwTgXVNqtr5IuO6ft+sON8stq4Zvhnb7YqdIuOmisGsztxHYGbWAO4jKImy1Le7Jt/MRnIi6JGy1Le7Jt/MWvlmcY+Upb7dNflm1sqJoEfKUt/umnwza+VE0CNlqW93Tb6ZtXIi6JGy1Le7Jt/MWvlmcY+Upb7dNflm1sp9BGZmDdCXPgJJU4HvAU/Pf89XI+JDLeu8CrgBeDBfdH1EXJIqJivuA8vXsXTFJvZEMEXivBNn8ZeLj+7K+mXpUShLHGb9Mm4ikPR04A3AnJHrd/AH+zfAKRHxqKQB4PuSbo6Iu1rWuzMiFhUL23rhA8vXce1d/7b39Z6Iva/b/XEvsn5ZehTKEodZP3Vys/gG4EzgSeDXI77GFJlH85cD+Ve1rkM13NIVm5ItL0uPQlniMOunTi4NDUbEaycyuKQpwCrgCOB/RsSKNqudJOleYCvw3oi4r804S4AlALNnz55IKDYBe0a5f9SN5WXpUShLHGb91MkZwQ8ljX5ReAwRsScijgUGgRMkzWtZZTXwvIh4CfBpYPko41wdEUMRMTRjxoyJhGITMEVKtrwsPQplicOsn0ZNBJLWSVoLvAJYLWmjpLUjlncsInYAdwCvbVn+8PDlo4i4CRiQdGjRjbA0zjtxVrLlZelRKEscZv001qWhSd3AlTQD2B0ROyRNA14DfKxlnecCv4iIkHQCWWLaPpnfa90zfIO30yqgIuuXpUehLHGY9dO4fQSS/iEi3jTesjafOwb4IjCF7A/8soi4RNLbASLiKkl/AryD7Eb0LuDCiPjhWOO6j8DMrLjJ9hEc1TLYFOD48T4UEWuB+W2WXzXi5yuAKzqIwczMEhk1EUi6GHg/ME3Sw8OLgSeAq3sQW+2kbFwq2viVatwyTHqTal9U1tplcNslsHMzHDwIp34Qjjmn31FZiYyaCCLiUuBSSZdGxMU9jKmWUjYuFW38SjVuGSa9SbUvKmvtMvjGu2F3Xg67c1P2GpwMbK+xqoaOk3Qc8JXhn0d+9TDGWkjZuFS0wSvVuGWY9CbVvqis2y7ZlwSG7d6VLTfLjXWP4G/z71OBIeBesktDxwAryMpKrUMpG5eKNnilGrcMk96k2heVtXNzseXWSKOeEUTEqyPi1cDPgePyhq7jyW4AP9CrAOsiZeNS0QavVOOWYdKbVPuisg4eLLbcGqmTzuL/EhHrhl9ExI+AY9OFVE8pG5eKNnilGrcMk96k2heVdeoHYaAluQ5My5ab5TopH90g6XPAtWQPjXsjsCFpVDWUsnGpaONXqnHLMOlNqn1RWcM3hF01ZGPopKFsKlnT18n5ou8BV0bE44lja8sNZWZmxU2qoSz/g//J/Msapmitvyd5sVG5n6G0xmooWxYR50haR5t5BCLimKSRWd8VrfX3JC82KvczlNpYN4svyL8vAl7f5stqrmitvyd5sVG5n6HUxuosfij/8VSy6STv701IVhZFa/09yYuNyv0MpdZJ+egc4DOSfippmaR3SXL5aAMUrfX3JC82KvczlNq4iSAiPhgRpwDzgO8DF5FNP2k1V7TW35O82Kjcz1Bq41YNSfoAsAB4FrAGeC9wZ+K4rASK1vp7khcblfsZSq2TPoLVZBPHfBP4LnBXv3oIwH0EZmYTMdk+guMkHUj2kLnfBT4r6RcRUduHzqWqhS86bhmeq+++gJKqe01+3bevqMT7o5NLQ/OA3wFeSfYU0k3U+NJQqlr4ouOW4bn67gsoqbrX5Nd9+4rqwf7opGroY8CBwOXAi/Knktb2Dk+qWvii45bhufruCyiputfk1337iurB/ujk0tDpXfttFZCqFr7ouGV4rr77Akqq7jX5dd++onqwPzo5I2iUVLXwRcctw3P13RdQUnWvya/79hXVg/3hRNAiVS180XHL8Fx99wWUVN1r8uu+fUX1YH90Mh9Bo6SqhS86bhmeq+++gJKqe01+3bevqB7sj1H7CCR9gzZPHR0WEWd0LYoC3EdgZlbcRPsIPp4onsZKWZNfZOwy9CeYVcKNF8KqL0DsAU2B48+HRZ/oztgl6pUY6+mj3+1lIHWXsia/yNhl6E8wq4QbL4SV1+x7HXv2vZ5sMihZr8S4N4slHSnpq5LWS/rX4a9eBFcnKWvyi4xdhv4Es0pY9YViy4soWa9EJ1VDfw9cSfa8oVcDXwL+IWVQdZSyJr/I2GXoTzCrhNhTbHkRJeuV6CQRTIuI28huLP88Ij4MnJI2rPpJWZNfZOwy9CeYVYKmFFteRMl6JTpJBI9L+i3gfkl/Iun3geckjqt2UtbkFxm7DP0JZpVw/PnFlhdRsl6JTvoI3gM8A3g38BGys4H/ljKoOkpZk19k7DL0J5hVwvAN4RRVQyXrlRh3PoK9K0oHARERj6QNaWzuIzAzK26sPoJOqoaGJK0D1gLrJN0r6fgOPjdV0t35+vdJ+h9t1pGkyyU9IGmtpOM62SAzM+ueTi4NfR747xFxJ4CkV5BVEh0zzud+A5wSEY9KGgC+L+nmiLhrxDqvA47Mv04kq046seA2jKtoI1cVJ2Mp0iRWZPuquC+SNuoUaTBKGUeqsUvU5JRMkW1swv6gs0TwyHASAIiI70sa9/JQZNecHs1fDuRfrdehzgS+lK97l6Tpkg6LiIc6C398RRu5qjgZS5EmsSLbV8V9kbRRp0iDUco4Uo1dsianJIpsYxP2R66TqqG7JX1G0qskvVLS/wLukHTceJdyJE2RdA/wS+DbEbGiZZWZZDOeDducL+uaoo1cVZyMpUiTWJHtq+K+SNqoU6TBKGUcqcYuWZNTEkW2sQn7I9fJGcGx+fcPtSx/Odm/8EftKYiIPcCxkqYDX5M0LyJ+NGKVdsXrT7l7LWkJsARg9uzZHYS8T9FGripOxlKkSazI9lVxXyRt1CnSYJQyjlRjl6zJKYki29iE/ZEb94wgn5pytK+OGssiYgdwB/Dalrc2AyML2AeBrW0+f3VEDEXE0IwZMzr5lXsVbeSq4mQsRZrEimxfFfdF0kadIg1GKeNINXbJmpySKLKNTdgfuU6qhn5b0jWSbs5fv1jS2zr43Iz8TABJ04DXAD9uWe3rwJvz6qGXATu7eX8AijdyVXEyliJNYkW2r4r7ImmjTpEGo5RxpBq7ZE1OSRTZxibsj1wnl4a+QFYl9Of5658A/whcM9oHcocBX5Q0hSzhLIuIGyW9HSAirgJuAhYCDwCPAW8pugHjKdrIVcXJWIo0iRXZvirui6SNOkUajFLGkWrskjU5JVFkG5uwP3LjNpRJ+peIeKmkNRExP192T0QcO+YHE3FDmZlZcROdmGbYryUdQn4Td/gSThfjK51K1s5bb1SxBj1lzFXsZyjLcSmRThLBhWTX8l8o6QfADODspFH1USVr5603qliDnjLmKvYzlOW4lEwnVUOrgVeSlYv+MXBURKxNHVi/VLJ23nqjijXoKWOuYj9DWY5LyXRSNfQHZHMS3AcsBv6xzs8EqmTtvPVGFWvQU8ZcxX6GshyXkumks/gvIuKR/BlDpwFfJHsmUC1VsnbeeqOKNegpY65iP0NZjkvJdJIIhq+TnA5cGRE3AE9LF1J/VbJ23nqjijXoKWOuYj9DWY5LyXSSCLZI+gxwDnCTpKd3+LlKWjx/JpeedTQzp09DwMzp07j0rKN9o9iym4mvvxwOngUo+/76y0evQe903arGnGobU+67shyXkumkj+AZZI+GWBcR90s6DDg6Ir7ViwBbuY/AzKy4SfURRMRjwPUjXj8EdPUxEGa1VGTugrKoYsxl6QsoSxwT0EkfgZkVVWTugrKoYsxl6QsoSxwTVNtr/WZ9VWTugrKoYsxl6QsoSxwT5ERglkKRuQvKoooxl6UvoCxxTJATgVkKReYuKIsqxlyWvoCyxDFBTgRmKRSZu6AsqhhzWfoCyhLHBDkRmKWw6BMw9LZ9/5rWlOx1WW+6QjVjLktfQFnimKBx+wjKxn0EZmbFTXY+ArM0qlh3nTLmVDX8VdzP1lNOBNYfVay7Thlzqhr+Ku5n6znfI7D+qGLddcqYU9XwV3E/W885EVh/VLHuOmXMqWr4q7ifreecCKw/qlh3nTLmVDX8VdzP1nNOBNYfVay7Thlzqhr+Ku5n6zknAuuPKtZdp4w5VQ1/Ffez9Zz7CMzMGmCsPgKfEZitXQafnAcfnp59X7usP+OmisNsHO4jsGZLVWdfdFzX+1sf+YzAmi1VnX3RcV3vb33kRGDNlqrOvui4rve3PnIisGZLVWdfdFzX+1sfORFYs6Wqsy86ruv9rY+cCKzZUtXZFx3X9f7WR+4jMDNrgL70EUiaJek7kjZIuk/SBW3WeZWknZLuyb98Hmxm1mMp+wieBP40IlZLOhBYJenbEbG+Zb07I2JRwjisl6o4CUqRmKu4fWXhfVdayRJBRDwEPJT//IikDcBMoDURWF1UsSmqSMxV3L6y8L4rtZ7cLJY0B5gPrGjz9kmS7pV0s6SjehGPJVLFpqgiMVdx+8rC+67Ukj9iQtKzgH8C3hMRD7e8vRp4XkQ8KmkhsBw4ss0YS4AlALNnz04csU1YFZuiisRcxe0rC++7Ukt6RiBpgCwJfDkirm99PyIejohH859vAgYkHdpmvasjYigihmbMmJEyZJuMKjZFFYm5ittXFt53pZayakjANcCGiGj7UHVJz83XQ9IJeTzbU8VkiVWxKapIzFXcvrLwviu1lJeGFgBvAtZJuidf9n5gNkBEXAWcDbxD0pPALuDcqFpjg+0zfNOvSpUhRWKu4vaVhfddqbmhzMysAcZqKPN8BE3keu793XghrPoCxJ5sisjjz5/8FJFmFeJE0DSu597fjRfCymv2vY49+147GVhD+KFzTeN67v2t+kKx5WY15ETQNK7n3l/sKbbcrIacCJrG9dz705Riy81qyImgaVzPvb/jzy+23KyGnAiaxhOg7G/RJ2DobfvOADQle+0bxdYg7iMwM2sA9xEktHzNFi67dSNbd+zi8OnTuOi0uSyeP7PfYXVPE3oOmrCNZeD9XFpOBJOwfM0WLr5+Hbt2ZxUmW3bs4uLr1wHUIxk0oeegCdtYBt7PpeZ7BJNw2a0b9yaBYbt27+GyWzf2KaIua0LPQRO2sQy8n0vNiWAStu7YVWh55TSh56AJ21gG3s+l5kQwCYdPn1ZoeeU0oeegCdtYBt7PpeZEMAkXnTaXaQP7Nx5NG5jCRafN7VNEXdaEnoMmbGMZeD+Xmm8WT8LwDeHaVg014RnyTdjGMvB+LjX3EZiZNcBYfQS+NGRWd2uXwSfnwYenZ9/XLqvG2NYzvjRkVmcp6/fdG1AbPiMwq7OU9fvuDagNJwKzOktZv+/egNpwIjCrs5T1++4NqA0nArM6S1m/796A2nAiMKuzlPNPeG6L2nAfgZlZA7iPwMzMRuVEYGbWcE4EZmYN50RgZtZwTgRmZg3nRGBm1nBOBGZmDedEYGbWcMkSgaRZkr4jaYOk+yRd0GYdSbpc0gOS1ko6LlU8ZmbWXsozgieBP42IFwEvA94p6cUt67wOODL/WgJcmTAemwhPPGJWe8kSQUQ8FBGr858fATYArZP5ngl8KTJ3AdMlHZYqJitoeOKRnZuA2DfxiJOBWa305B6BpDnAfGBFy1szgU0jXm/mqcnC+sUTj5g1QvJEIOlZwD8B74mIh1vfbvORpzwFT9ISSSslrdy2bVuKMK0dTzxi1ghJE4GkAbIk8OWIuL7NKpuBWSNeDwJbW1eKiKsjYigihmbMmJEmWHsqTzxi1ggpq4YEXANsiIhPjLLa14E359VDLwN2RsRDqWKygjzxiFkjHJBw7AXAm4B1ku7Jl70fmA0QEVcBNwELgQeAx4C3JIzHihqeYOS2S7LLQQcPZknAE4+Y1YonpjEzawBPTGNmZqNyIjAzazgnAjOzhnMiMDNrOCcCM7OGq1zVkKRtwM/7HUcbhwL/3u8gEqr79kH9t9HbV32T2cbnRUTbjtzKJYKykrRytNKsOqj79kH9t9HbV32pttGXhszMGs6JwMys4ZwIuufqfgeQWN23D+q/jd6+6kuyjb5HYGbWcD4jMDNrOCeCgiRNkbRG0o1t3nuVpJ2S7sm/Kve8Zkk/k7Quj/8pT/fLHxl+uaQHJK2VdFw/4pyMDrax0sdR0nRJX5X0Y0kbJJ3U8n6lj2EH21f14zd3ROz3SHpY0nta1unqMUz5GOq6uoBs/uWDRnn/zohY1MN4Unh1RIxWq/w64Mj860Tgyvx71Yy1jVDt4/h3wC0RcbakpwHPaHm/6sdwvO2DCh+/iNgIHAvZPzyBLcDXWlbr6jH0GUEBkgaB04HP9TuWPjoT+FJk7gKmSzqs30FZRtJBwMlkk0IREU9ExI6W1Sp7DDvcvjo5FfhpRLQ20Xb1GDoRFPMp4H3Af4yxzkmS7pV0s6SjehRXNwXwLUmrJC1p8/5MYNOI15vzZVUy3jZCdY/jC4BtwN/nlzA/J+mZLetU+Rh2sn1Q3ePX6lxgaZvlXT2GTgQdkrQI+GVErBpjtdVkbdwvAT4NLO9JcN21ICKOIzv1fKekk1veV5vPVK30bLxtrPJxPAA4DrgyIuYDvwb+rGWdKh/DTravysdvr/yy1xnAV9q93WbZhI+hE0HnFgBnSPoZcB1wiqRrR64QEQ9HxKP5zzcBA5IO7XmkkxARW/PvvyS7LnlCyyqbgVkjXg8CW3sTXXeMt40VP46bgc0RsSJ//VWyP5yt61T1GI67fRU/fiO9DlgdEb9o815Xj6ETQYci4uKIGIyIOWSna7dHxBtHriPpuZKU/3wC2f7d3vNgJ0jSMyUdOPwz8HvAj1pW+zrw5rxq4WXAzoh4qMehTlgn21jl4xgR/xfYJGluvuhUYH3LapU9hp1sX5WPX4vzaH9ZCLp8DF01NEmS3g4QEVcBZwPvkPQksAs4N6rVsffbwNfy/4cOAP53RNzSso03AQuBB4DHgLf0KdaJ6mQbq34c3wV8Ob+08K/AW2p2DMfbvqofPyQ9A/hd4I9HLEt2DN1ZbGbWcL40ZGbWcE4EZmYN50RgZtZwTgRmZg3nRGBm1nBOBGYF5U+3HO3ps09Z3oXft1jSi0e8vkNSrefmtd5yIjArv8XAi8ddy2yCnAisdvLu4W/mDx37kaT/mi8/XtJ384fN3Tr8tMb8X9ifkvTDfP0T8uUn5MvW5N/njvV728TweUn/kn/+zHz5+ZKul3SLpPsl/c2Iz7xN0k/yeD4r6QpJLyd73sxlyp5N/8J89T+QdHe+/u90addZQ7mz2OrotcDWiDgdQNLBkgbIHkB2ZkRsy5PDXwFvzT/zzIh4ef4Aus8D84AfAydHxJOSXgP8NfCGDmP4c7LHkLxV0nTgbkn/J3/vWGA+8Btgo6RPA3uAvyB7bs4jwO3AvRHxQ0lfB26MiK/m2wNwQEScIGkh8CHgNRPZUWbgRGD1tA74uKSPkf0BvVPSPLI/7t/O/5BOAUY+m2UpQER8T9JB+R/vA4EvSjqS7MmOAwVi+D2yhxS+N389FZid/3xbROwEkLQeeB5wKPDdiPhVvvwrwH8eY/zr8++rgDkF4jJ7CicCq52I+Imk48mexXKppG+RPWX0vog4abSPtXn9EeA7EfH7kuYAdxQIQ8Ab8tmm9i2UTiQ7Exi2h+z/w3aPFR7L8BjDnzebMN8jsNqRdDjwWERcC3yc7HLLRmCG8vltJQ1o/wlLhu8jvILsSY47gYPJpgkEOL9gGLcC7xrxFMz546x/N/BKSf9J0gHsfwnqEbKzE7Mk/C8Jq6OjyW6u/gewG3hHRDwh6WzgckkHk/23/yngvvwz/0/SD8nmoh6+b/A3ZJeGLiS7Zl/ER/Lx1+bJ4GfAqHPoRsQWSX8NrCB7rvx6YGf+9nXAZyW9m+zJmmZd5aePWuNJugN4b0Ss7HMcz4qIR/Mzgq8Bn4+I1knLzbrOl4bMyuPDku4hmyjnQSo6xaJVj88IzMwazmcEZmYN50RgZtZwTgRmZg3nRGBm1nBOBGZmDedEYGbWcP8fHDGcEmDqq8wAAAAASUVORK5CYII=\n"
     },
     "metadata": {
      "needs_background": "light"
     }
    }
   ],
   "source": [
    "plt.scatter(df[:50]['sepal length'], df[:50]['sepal width'], label='0')\n",
    "plt.scatter(df[50:100]['sepal length'], df[50:100]['sepal width'], label='1')\n",
    "plt.xlabel('sepal length')\n",
    "plt.ylabel('sepal width')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = np.array(df.iloc[:100, [0, 1, -1]])\n",
    "X, y = data[:,:-1], data[:,-1]\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "假设`X_test`为待标记的数据样本，`X_train`为已标记的数据样本，算法原理的伪代码如下：  \n",
    "* 遍历`X_train`中的所有样本，计算每个样本与`X_test`的距离，并把距离保存在`Distance`数组中；\n",
    "* 对`Distance`数组进行排序，取距离最近的k个点，记为`X_knn`；\n",
    "* 在`X_knn`中统计每个类别的个数，即`class0`在`X_knn`中有几个样本，`class1`在`X_knn`中有几个样本等；\n",
    "* 待标记样本的实例，就是在`X_knn`中样本个数最多的那个类别。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "class KNN:\n",
    "    def __init__(self, X_train, y_train, n_neighbors=3, p=2):\n",
    "        \"\"\"\n",
    "        parameter: n_neighbors 临近点个数\n",
    "        parameter: p 距离度量\n",
    "        \"\"\"\n",
    "        self.n = n_neighbors\n",
    "        self.p = p\n",
    "        self.X_train = X_train\n",
    "        self.y_train = y_train\n",
    "\n",
    "    def predict(self, X):\n",
    "        # 取出n个点\n",
    "        knn_list = []\n",
    "        for i in range(self.n):\n",
    "            dist = np.linalg.norm(X - self.X_train[i], ord=self.p)\n",
    "            knn_list.append((dist, self.y_train[i]))\n",
    "\n",
    "        for i in range(self.n, len(self.X_train)):\n",
    "            max_index = knn_list.index(max(knn_list, key=lambda x: x[0]))\n",
    "            dist = np.linalg.norm(X - self.X_train[i], ord=self.p)\n",
    "            if knn_list[max_index][0] > dist:\n",
    "                knn_list[max_index] = (dist, self.y_train[i])\n",
    "\n",
    "        # 统计\n",
    "        knn = [k[-1] for k in knn_list]\n",
    "        count_pairs = Counter(knn)\n",
    "#         max_count = sorted(count_pairs, key=lambda x: x)[-1]\n",
    "        max_count = sorted(count_pairs.items(), key=lambda x: x[1])[-1][0]\n",
    "        return max_count\n",
    "\n",
    "    def score(self, X_test, y_test):\n",
    "        right_count = 0\n",
    "        n = 10\n",
    "        for X, y in zip(X_test, y_test):\n",
    "            label = self.predict(X)\n",
    "            if label == y:\n",
    "                right_count += 1\n",
    "        return right_count / len(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "clf = KNN(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "1.0"
     },
     "metadata": {},
     "execution_count": 12
    }
   ],
   "source": [
    "clf.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": "Test Point: 1.0\n"
    }
   ],
   "source": [
    "test_point = [6.0, 3.0]\n",
    "print('Test Point: {}'.format(clf.predict(test_point)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": "<matplotlib.legend.Legend at 0x1e1b7606988>"
     },
     "metadata": {},
     "execution_count": 14
    },
    {
     "output_type": "display_data",
     "data": {
      "text/plain": "<Figure size 432x288 with 1 Axes>",
      "image/svg+xml": "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"no\"?>\r\n<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\r\n  \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\r\n<!-- Created with matplotlib (https://matplotlib.org/) -->\r\n<svg height=\"264.348196pt\" version=\"1.1\" viewBox=\"0 0 385.78125 264.348196\" width=\"385.78125pt\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\r\n <defs>\r\n  <style type=\"text/css\">\r\n*{stroke-linecap:butt;stroke-linejoin:round;}\r\n  </style>\r\n </defs>\r\n <g id=\"figure_1\">\r\n  <g id=\"patch_1\">\r\n   <path d=\"M 0 264.348196 \r\nL 385.78125 264.348196 \r\nL 385.78125 0 \r\nL 0 0 \r\nz\r\n\" style=\"fill:none;\"/>\r\n  </g>\r\n  <g id=\"axes_1\">\r\n   <g id=\"patch_2\">\r\n    <path d=\"M 43.78125 226.791946 \r\nL 378.58125 226.791946 \r\nL 378.58125 9.351946 \r\nL 43.78125 9.351946 \r\nz\r\n\" style=\"fill:#ffffff;\"/>\r\n   </g>\r\n   <g id=\"PathCollection_1\">\r\n    <defs>\r\n     <path d=\"M 0 3 \r\nC 0.795609 3 1.55874 2.683901 2.12132 2.12132 \r\nC 2.683901 1.55874 3 0.795609 3 0 \r\nC 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \r\nC 1.55874 -2.683901 0.795609 -3 0 -3 \r\nC -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \r\nC -2.683901 -1.55874 -3 -0.795609 -3 0 \r\nC -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \r\nC -1.55874 2.683901 -0.795609 3 0 3 \r\nz\r\n\" id=\"me5904ac1a2\" style=\"stroke:#1f77b4;\"/>\r\n    </defs>\r\n    <g clip-path=\"url(#pc48046d1f2)\">\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"149.18125\" xlink:href=\"#me5904ac1a2\" y=\"93.362855\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"126.635795\" xlink:href=\"#me5904ac1a2\" y=\"134.544673\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"104.090341\" xlink:href=\"#me5904ac1a2\" y=\"118.071946\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"92.817614\" xlink:href=\"#me5904ac1a2\" y=\"126.30831\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"137.908523\" xlink:href=\"#me5904ac1a2\" y=\"85.126491\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"182.999432\" xlink:href=\"#me5904ac1a2\" y=\"60.417401\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"92.817614\" xlink:href=\"#me5904ac1a2\" y=\"101.599219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"137.908523\" xlink:href=\"#me5904ac1a2\" y=\"101.599219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"70.272159\" xlink:href=\"#me5904ac1a2\" y=\"142.781037\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"126.635795\" xlink:href=\"#me5904ac1a2\" y=\"126.30831\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"182.999432\" xlink:href=\"#me5904ac1a2\" y=\"76.890128\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"115.363068\" xlink:href=\"#me5904ac1a2\" y=\"101.599219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"115.363068\" xlink:href=\"#me5904ac1a2\" y=\"134.544673\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"58.999432\" xlink:href=\"#me5904ac1a2\" y=\"134.544673\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"228.090341\" xlink:href=\"#me5904ac1a2\" y=\"52.181037\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"216.817614\" xlink:href=\"#me5904ac1a2\" y=\"19.235582\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"182.999432\" xlink:href=\"#me5904ac1a2\" y=\"60.417401\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"149.18125\" xlink:href=\"#me5904ac1a2\" y=\"93.362855\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"216.817614\" xlink:href=\"#me5904ac1a2\" y=\"68.653764\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"149.18125\" xlink:href=\"#me5904ac1a2\" y=\"68.653764\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"182.999432\" xlink:href=\"#me5904ac1a2\" y=\"101.599219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"149.18125\" xlink:href=\"#me5904ac1a2\" y=\"76.890128\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"92.817614\" xlink:href=\"#me5904ac1a2\" y=\"85.126491\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"149.18125\" xlink:href=\"#me5904ac1a2\" y=\"109.835582\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"115.363068\" xlink:href=\"#me5904ac1a2\" y=\"101.599219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"137.908523\" xlink:href=\"#me5904ac1a2\" y=\"134.544673\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"137.908523\" xlink:href=\"#me5904ac1a2\" y=\"101.599219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"160.453977\" xlink:href=\"#me5904ac1a2\" y=\"93.362855\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"160.453977\" xlink:href=\"#me5904ac1a2\" y=\"101.599219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"104.090341\" xlink:href=\"#me5904ac1a2\" y=\"118.071946\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"115.363068\" xlink:href=\"#me5904ac1a2\" y=\"126.30831\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"182.999432\" xlink:href=\"#me5904ac1a2\" y=\"101.599219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"160.453977\" xlink:href=\"#me5904ac1a2\" y=\"43.944673\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"194.272159\" xlink:href=\"#me5904ac1a2\" y=\"35.70831\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"126.635795\" xlink:href=\"#me5904ac1a2\" y=\"126.30831\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"137.908523\" xlink:href=\"#me5904ac1a2\" y=\"118.071946\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"194.272159\" xlink:href=\"#me5904ac1a2\" y=\"93.362855\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"126.635795\" xlink:href=\"#me5904ac1a2\" y=\"85.126491\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"70.272159\" xlink:href=\"#me5904ac1a2\" y=\"134.544673\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"149.18125\" xlink:href=\"#me5904ac1a2\" y=\"101.599219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"137.908523\" xlink:href=\"#me5904ac1a2\" y=\"93.362855\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"81.544886\" xlink:href=\"#me5904ac1a2\" y=\"192.199219\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"70.272159\" xlink:href=\"#me5904ac1a2\" y=\"118.071946\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"137.908523\" xlink:href=\"#me5904ac1a2\" y=\"93.362855\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"149.18125\" xlink:href=\"#me5904ac1a2\" y=\"68.653764\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"115.363068\" xlink:href=\"#me5904ac1a2\" y=\"134.544673\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"149.18125\" xlink:href=\"#me5904ac1a2\" y=\"68.653764\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"92.817614\" xlink:href=\"#me5904ac1a2\" y=\"118.071946\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"171.726705\" xlink:href=\"#me5904ac1a2\" y=\"76.890128\"/>\r\n     <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"137.908523\" xlink:href=\"#me5904ac1a2\" y=\"109.835582\"/>\r\n    </g>\r\n   </g>\r\n   <g id=\"PathCollection_2\">\r\n    <defs>\r\n     <path d=\"M 0 3 \r\nC 0.795609 3 1.55874 2.683901 2.12132 2.12132 \r\nC 2.683901 1.55874 3 0.795609 3 0 \r\nC 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \r\nC 1.55874 -2.683901 0.795609 -3 0 -3 \r\nC -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \r\nC -2.683901 -1.55874 -3 -0.795609 -3 0 \r\nC -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \r\nC -1.55874 2.683901 -0.795609 3 0 3 \r\nz\r\n\" id=\"m5fb07bbd4d\" style=\"stroke:#ff7f0e;\"/>\r\n    </defs>\r\n    <g clip-path=\"url(#pc48046d1f2)\">\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"363.363068\" xlink:href=\"#m5fb07bbd4d\" y=\"118.071946\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"295.726705\" xlink:href=\"#m5fb07bbd4d\" y=\"118.071946\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"352.090341\" xlink:href=\"#m5fb07bbd4d\" y=\"126.30831\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"194.272159\" xlink:href=\"#m5fb07bbd4d\" y=\"192.199219\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"306.999432\" xlink:href=\"#m5fb07bbd4d\" y=\"151.017401\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"216.817614\" xlink:href=\"#m5fb07bbd4d\" y=\"151.017401\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"284.453977\" xlink:href=\"#m5fb07bbd4d\" y=\"109.835582\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"126.635795\" xlink:href=\"#m5fb07bbd4d\" y=\"183.962855\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"318.272159\" xlink:href=\"#m5fb07bbd4d\" y=\"142.781037\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"160.453977\" xlink:href=\"#m5fb07bbd4d\" y=\"159.253764\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"137.908523\" xlink:href=\"#m5fb07bbd4d\" y=\"216.90831\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"239.363068\" xlink:href=\"#m5fb07bbd4d\" y=\"134.544673\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"250.635795\" xlink:href=\"#m5fb07bbd4d\" y=\"200.435582\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"261.908523\" xlink:href=\"#m5fb07bbd4d\" y=\"142.781037\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"205.544886\" xlink:href=\"#m5fb07bbd4d\" y=\"142.781037\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"329.544886\" xlink:href=\"#m5fb07bbd4d\" y=\"126.30831\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"205.544886\" xlink:href=\"#m5fb07bbd4d\" y=\"134.544673\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"228.090341\" xlink:href=\"#m5fb07bbd4d\" y=\"159.253764\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"273.18125\" xlink:href=\"#m5fb07bbd4d\" y=\"200.435582\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"205.544886\" xlink:href=\"#m5fb07bbd4d\" y=\"175.726491\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"239.363068\" xlink:href=\"#m5fb07bbd4d\" y=\"118.071946\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"261.908523\" xlink:href=\"#m5fb07bbd4d\" y=\"151.017401\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"284.453977\" xlink:href=\"#m5fb07bbd4d\" y=\"175.726491\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"261.908523\" xlink:href=\"#m5fb07bbd4d\" y=\"151.017401\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"295.726705\" xlink:href=\"#m5fb07bbd4d\" y=\"142.781037\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"318.272159\" xlink:href=\"#m5fb07bbd4d\" y=\"134.544673\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"340.817614\" xlink:href=\"#m5fb07bbd4d\" y=\"151.017401\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"329.544886\" xlink:href=\"#m5fb07bbd4d\" y=\"134.544673\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"250.635795\" xlink:href=\"#m5fb07bbd4d\" y=\"142.781037\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"216.817614\" xlink:href=\"#m5fb07bbd4d\" y=\"167.490128\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"194.272159\" xlink:href=\"#m5fb07bbd4d\" y=\"183.962855\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"194.272159\" xlink:href=\"#m5fb07bbd4d\" y=\"183.962855\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"228.090341\" xlink:href=\"#m5fb07bbd4d\" y=\"159.253764\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"250.635795\" xlink:href=\"#m5fb07bbd4d\" y=\"159.253764\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"182.999432\" xlink:href=\"#m5fb07bbd4d\" y=\"134.544673\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"250.635795\" xlink:href=\"#m5fb07bbd4d\" y=\"101.599219\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"329.544886\" xlink:href=\"#m5fb07bbd4d\" y=\"126.30831\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"284.453977\" xlink:href=\"#m5fb07bbd4d\" y=\"192.199219\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"205.544886\" xlink:href=\"#m5fb07bbd4d\" y=\"134.544673\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"194.272159\" xlink:href=\"#m5fb07bbd4d\" y=\"175.726491\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"194.272159\" xlink:href=\"#m5fb07bbd4d\" y=\"167.490128\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"261.908523\" xlink:href=\"#m5fb07bbd4d\" y=\"134.544673\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"228.090341\" xlink:href=\"#m5fb07bbd4d\" y=\"167.490128\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"137.908523\" xlink:href=\"#m5fb07bbd4d\" y=\"192.199219\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"205.544886\" xlink:href=\"#m5fb07bbd4d\" y=\"159.253764\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"216.817614\" xlink:href=\"#m5fb07bbd4d\" y=\"134.544673\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"216.817614\" xlink:href=\"#m5fb07bbd4d\" y=\"142.781037\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"273.18125\" xlink:href=\"#m5fb07bbd4d\" y=\"142.781037\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"149.18125\" xlink:href=\"#m5fb07bbd4d\" y=\"175.726491\"/>\r\n     <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"216.817614\" xlink:href=\"#m5fb07bbd4d\" y=\"151.017401\"/>\r\n    </g>\r\n   </g>\r\n   <g id=\"matplotlib.axis_1\">\r\n    <g id=\"xtick_1\">\r\n     <g id=\"line2d_1\">\r\n      <defs>\r\n       <path d=\"M 0 0 \r\nL 0 3.5 \r\n\" id=\"m040b725831\" style=\"stroke:#000000;stroke-width:0.8;\"/>\r\n      </defs>\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"81.544886\" xlink:href=\"#m040b725831\" y=\"226.791946\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_1\">\r\n      <!-- 4.5 -->\r\n      <defs>\r\n       <path d=\"M 37.796875 64.3125 \r\nL 12.890625 25.390625 \r\nL 37.796875 25.390625 \r\nz\r\nM 35.203125 72.90625 \r\nL 47.609375 72.90625 \r\nL 47.609375 25.390625 \r\nL 58.015625 25.390625 \r\nL 58.015625 17.1875 \r\nL 47.609375 17.1875 \r\nL 47.609375 0 \r\nL 37.796875 0 \r\nL 37.796875 17.1875 \r\nL 4.890625 17.1875 \r\nL 4.890625 26.703125 \r\nz\r\n\" id=\"DejaVuSans-52\"/>\r\n       <path d=\"M 10.6875 12.40625 \r\nL 21 12.40625 \r\nL 21 0 \r\nL 10.6875 0 \r\nz\r\n\" id=\"DejaVuSans-46\"/>\r\n       <path d=\"M 10.796875 72.90625 \r\nL 49.515625 72.90625 \r\nL 49.515625 64.59375 \r\nL 19.828125 64.59375 \r\nL 19.828125 46.734375 \r\nQ 21.96875 47.46875 24.109375 47.828125 \r\nQ 26.265625 48.1875 28.421875 48.1875 \r\nQ 40.625 48.1875 47.75 41.5 \r\nQ 54.890625 34.8125 54.890625 23.390625 \r\nQ 54.890625 11.625 47.5625 5.09375 \r\nQ 40.234375 -1.421875 26.90625 -1.421875 \r\nQ 22.3125 -1.421875 17.546875 -0.640625 \r\nQ 12.796875 0.140625 7.71875 1.703125 \r\nL 7.71875 11.625 \r\nQ 12.109375 9.234375 16.796875 8.0625 \r\nQ 21.484375 6.890625 26.703125 6.890625 \r\nQ 35.15625 6.890625 40.078125 11.328125 \r\nQ 45.015625 15.765625 45.015625 23.390625 \r\nQ 45.015625 31 40.078125 35.4375 \r\nQ 35.15625 39.890625 26.703125 39.890625 \r\nQ 22.75 39.890625 18.8125 39.015625 \r\nQ 14.890625 38.140625 10.796875 36.28125 \r\nz\r\n\" id=\"DejaVuSans-53\"/>\r\n      </defs>\r\n      <g transform=\"translate(73.593324 241.390384)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-52\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"xtick_2\">\r\n     <g id=\"line2d_2\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"137.908523\" xlink:href=\"#m040b725831\" y=\"226.791946\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_2\">\r\n      <!-- 5.0 -->\r\n      <defs>\r\n       <path d=\"M 31.78125 66.40625 \r\nQ 24.171875 66.40625 20.328125 58.90625 \r\nQ 16.5 51.421875 16.5 36.375 \r\nQ 16.5 21.390625 20.328125 13.890625 \r\nQ 24.171875 6.390625 31.78125 6.390625 \r\nQ 39.453125 6.390625 43.28125 13.890625 \r\nQ 47.125 21.390625 47.125 36.375 \r\nQ 47.125 51.421875 43.28125 58.90625 \r\nQ 39.453125 66.40625 31.78125 66.40625 \r\nz\r\nM 31.78125 74.21875 \r\nQ 44.046875 74.21875 50.515625 64.515625 \r\nQ 56.984375 54.828125 56.984375 36.375 \r\nQ 56.984375 17.96875 50.515625 8.265625 \r\nQ 44.046875 -1.421875 31.78125 -1.421875 \r\nQ 19.53125 -1.421875 13.0625 8.265625 \r\nQ 6.59375 17.96875 6.59375 36.375 \r\nQ 6.59375 54.828125 13.0625 64.515625 \r\nQ 19.53125 74.21875 31.78125 74.21875 \r\nz\r\n\" id=\"DejaVuSans-48\"/>\r\n      </defs>\r\n      <g transform=\"translate(129.95696 241.390384)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-53\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"xtick_3\">\r\n     <g id=\"line2d_3\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"194.272159\" xlink:href=\"#m040b725831\" y=\"226.791946\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_3\">\r\n      <!-- 5.5 -->\r\n      <g transform=\"translate(186.320597 241.390384)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-53\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"xtick_4\">\r\n     <g id=\"line2d_4\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"250.635795\" xlink:href=\"#m040b725831\" y=\"226.791946\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_4\">\r\n      <!-- 6.0 -->\r\n      <defs>\r\n       <path d=\"M 33.015625 40.375 \r\nQ 26.375 40.375 22.484375 35.828125 \r\nQ 18.609375 31.296875 18.609375 23.390625 \r\nQ 18.609375 15.53125 22.484375 10.953125 \r\nQ 26.375 6.390625 33.015625 6.390625 \r\nQ 39.65625 6.390625 43.53125 10.953125 \r\nQ 47.40625 15.53125 47.40625 23.390625 \r\nQ 47.40625 31.296875 43.53125 35.828125 \r\nQ 39.65625 40.375 33.015625 40.375 \r\nz\r\nM 52.59375 71.296875 \r\nL 52.59375 62.3125 \r\nQ 48.875 64.0625 45.09375 64.984375 \r\nQ 41.3125 65.921875 37.59375 65.921875 \r\nQ 27.828125 65.921875 22.671875 59.328125 \r\nQ 17.53125 52.734375 16.796875 39.40625 \r\nQ 19.671875 43.65625 24.015625 45.921875 \r\nQ 28.375 48.1875 33.59375 48.1875 \r\nQ 44.578125 48.1875 50.953125 41.515625 \r\nQ 57.328125 34.859375 57.328125 23.390625 \r\nQ 57.328125 12.15625 50.6875 5.359375 \r\nQ 44.046875 -1.421875 33.015625 -1.421875 \r\nQ 20.359375 -1.421875 13.671875 8.265625 \r\nQ 6.984375 17.96875 6.984375 36.375 \r\nQ 6.984375 53.65625 15.1875 63.9375 \r\nQ 23.390625 74.21875 37.203125 74.21875 \r\nQ 40.921875 74.21875 44.703125 73.484375 \r\nQ 48.484375 72.75 52.59375 71.296875 \r\nz\r\n\" id=\"DejaVuSans-54\"/>\r\n      </defs>\r\n      <g transform=\"translate(242.684233 241.390384)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-54\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"xtick_5\">\r\n     <g id=\"line2d_5\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"306.999432\" xlink:href=\"#m040b725831\" y=\"226.791946\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_5\">\r\n      <!-- 6.5 -->\r\n      <g transform=\"translate(299.047869 241.390384)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-54\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"xtick_6\">\r\n     <g id=\"line2d_6\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"363.363068\" xlink:href=\"#m040b725831\" y=\"226.791946\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_6\">\r\n      <!-- 7.0 -->\r\n      <defs>\r\n       <path d=\"M 8.203125 72.90625 \r\nL 55.078125 72.90625 \r\nL 55.078125 68.703125 \r\nL 28.609375 0 \r\nL 18.3125 0 \r\nL 43.21875 64.59375 \r\nL 8.203125 64.59375 \r\nz\r\n\" id=\"DejaVuSans-55\"/>\r\n      </defs>\r\n      <g transform=\"translate(355.411506 241.390384)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-55\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"text_7\">\r\n     <!-- sepal length -->\r\n     <defs>\r\n      <path d=\"M 44.28125 53.078125 \r\nL 44.28125 44.578125 \r\nQ 40.484375 46.53125 36.375 47.5 \r\nQ 32.28125 48.484375 27.875 48.484375 \r\nQ 21.1875 48.484375 17.84375 46.4375 \r\nQ 14.5 44.390625 14.5 40.28125 \r\nQ 14.5 37.15625 16.890625 35.375 \r\nQ 19.28125 33.59375 26.515625 31.984375 \r\nL 29.59375 31.296875 \r\nQ 39.15625 29.25 43.1875 25.515625 \r\nQ 47.21875 21.78125 47.21875 15.09375 \r\nQ 47.21875 7.46875 41.1875 3.015625 \r\nQ 35.15625 -1.421875 24.609375 -1.421875 \r\nQ 20.21875 -1.421875 15.453125 -0.5625 \r\nQ 10.6875 0.296875 5.421875 2 \r\nL 5.421875 11.28125 \r\nQ 10.40625 8.6875 15.234375 7.390625 \r\nQ 20.0625 6.109375 24.8125 6.109375 \r\nQ 31.15625 6.109375 34.5625 8.28125 \r\nQ 37.984375 10.453125 37.984375 14.40625 \r\nQ 37.984375 18.0625 35.515625 20.015625 \r\nQ 33.0625 21.96875 24.703125 23.78125 \r\nL 21.578125 24.515625 \r\nQ 13.234375 26.265625 9.515625 29.90625 \r\nQ 5.8125 33.546875 5.8125 39.890625 \r\nQ 5.8125 47.609375 11.28125 51.796875 \r\nQ 16.75 56 26.8125 56 \r\nQ 31.78125 56 36.171875 55.265625 \r\nQ 40.578125 54.546875 44.28125 53.078125 \r\nz\r\n\" id=\"DejaVuSans-115\"/>\r\n      <path d=\"M 56.203125 29.59375 \r\nL 56.203125 25.203125 \r\nL 14.890625 25.203125 \r\nQ 15.484375 15.921875 20.484375 11.0625 \r\nQ 25.484375 6.203125 34.421875 6.203125 \r\nQ 39.59375 6.203125 44.453125 7.46875 \r\nQ 49.3125 8.734375 54.109375 11.28125 \r\nL 54.109375 2.78125 \r\nQ 49.265625 0.734375 44.1875 -0.34375 \r\nQ 39.109375 -1.421875 33.890625 -1.421875 \r\nQ 20.796875 -1.421875 13.15625 6.1875 \r\nQ 5.515625 13.8125 5.515625 26.8125 \r\nQ 5.515625 40.234375 12.765625 48.109375 \r\nQ 20.015625 56 32.328125 56 \r\nQ 43.359375 56 49.78125 48.890625 \r\nQ 56.203125 41.796875 56.203125 29.59375 \r\nz\r\nM 47.21875 32.234375 \r\nQ 47.125 39.59375 43.09375 43.984375 \r\nQ 39.0625 48.390625 32.421875 48.390625 \r\nQ 24.90625 48.390625 20.390625 44.140625 \r\nQ 15.875 39.890625 15.1875 32.171875 \r\nz\r\n\" id=\"DejaVuSans-101\"/>\r\n      <path d=\"M 18.109375 8.203125 \r\nL 18.109375 -20.796875 \r\nL 9.078125 -20.796875 \r\nL 9.078125 54.6875 \r\nL 18.109375 54.6875 \r\nL 18.109375 46.390625 \r\nQ 20.953125 51.265625 25.265625 53.625 \r\nQ 29.59375 56 35.59375 56 \r\nQ 45.5625 56 51.78125 48.09375 \r\nQ 58.015625 40.1875 58.015625 27.296875 \r\nQ 58.015625 14.40625 51.78125 6.484375 \r\nQ 45.5625 -1.421875 35.59375 -1.421875 \r\nQ 29.59375 -1.421875 25.265625 0.953125 \r\nQ 20.953125 3.328125 18.109375 8.203125 \r\nz\r\nM 48.6875 27.296875 \r\nQ 48.6875 37.203125 44.609375 42.84375 \r\nQ 40.53125 48.484375 33.40625 48.484375 \r\nQ 26.265625 48.484375 22.1875 42.84375 \r\nQ 18.109375 37.203125 18.109375 27.296875 \r\nQ 18.109375 17.390625 22.1875 11.75 \r\nQ 26.265625 6.109375 33.40625 6.109375 \r\nQ 40.53125 6.109375 44.609375 11.75 \r\nQ 48.6875 17.390625 48.6875 27.296875 \r\nz\r\n\" id=\"DejaVuSans-112\"/>\r\n      <path d=\"M 34.28125 27.484375 \r\nQ 23.390625 27.484375 19.1875 25 \r\nQ 14.984375 22.515625 14.984375 16.5 \r\nQ 14.984375 11.71875 18.140625 8.90625 \r\nQ 21.296875 6.109375 26.703125 6.109375 \r\nQ 34.1875 6.109375 38.703125 11.40625 \r\nQ 43.21875 16.703125 43.21875 25.484375 \r\nL 43.21875 27.484375 \r\nz\r\nM 52.203125 31.203125 \r\nL 52.203125 0 \r\nL 43.21875 0 \r\nL 43.21875 8.296875 \r\nQ 40.140625 3.328125 35.546875 0.953125 \r\nQ 30.953125 -1.421875 24.3125 -1.421875 \r\nQ 15.921875 -1.421875 10.953125 3.296875 \r\nQ 6 8.015625 6 15.921875 \r\nQ 6 25.140625 12.171875 29.828125 \r\nQ 18.359375 34.515625 30.609375 34.515625 \r\nL 43.21875 34.515625 \r\nL 43.21875 35.40625 \r\nQ 43.21875 41.609375 39.140625 45 \r\nQ 35.0625 48.390625 27.6875 48.390625 \r\nQ 23 48.390625 18.546875 47.265625 \r\nQ 14.109375 46.140625 10.015625 43.890625 \r\nL 10.015625 52.203125 \r\nQ 14.9375 54.109375 19.578125 55.046875 \r\nQ 24.21875 56 28.609375 56 \r\nQ 40.484375 56 46.34375 49.84375 \r\nQ 52.203125 43.703125 52.203125 31.203125 \r\nz\r\n\" id=\"DejaVuSans-97\"/>\r\n      <path d=\"M 9.421875 75.984375 \r\nL 18.40625 75.984375 \r\nL 18.40625 0 \r\nL 9.421875 0 \r\nz\r\n\" id=\"DejaVuSans-108\"/>\r\n      <path id=\"DejaVuSans-32\"/>\r\n      <path d=\"M 54.890625 33.015625 \r\nL 54.890625 0 \r\nL 45.90625 0 \r\nL 45.90625 32.71875 \r\nQ 45.90625 40.484375 42.875 44.328125 \r\nQ 39.84375 48.1875 33.796875 48.1875 \r\nQ 26.515625 48.1875 22.3125 43.546875 \r\nQ 18.109375 38.921875 18.109375 30.90625 \r\nL 18.109375 0 \r\nL 9.078125 0 \r\nL 9.078125 54.6875 \r\nL 18.109375 54.6875 \r\nL 18.109375 46.1875 \r\nQ 21.34375 51.125 25.703125 53.5625 \r\nQ 30.078125 56 35.796875 56 \r\nQ 45.21875 56 50.046875 50.171875 \r\nQ 54.890625 44.34375 54.890625 33.015625 \r\nz\r\n\" id=\"DejaVuSans-110\"/>\r\n      <path d=\"M 45.40625 27.984375 \r\nQ 45.40625 37.75 41.375 43.109375 \r\nQ 37.359375 48.484375 30.078125 48.484375 \r\nQ 22.859375 48.484375 18.828125 43.109375 \r\nQ 14.796875 37.75 14.796875 27.984375 \r\nQ 14.796875 18.265625 18.828125 12.890625 \r\nQ 22.859375 7.515625 30.078125 7.515625 \r\nQ 37.359375 7.515625 41.375 12.890625 \r\nQ 45.40625 18.265625 45.40625 27.984375 \r\nz\r\nM 54.390625 6.78125 \r\nQ 54.390625 -7.171875 48.1875 -13.984375 \r\nQ 42 -20.796875 29.203125 -20.796875 \r\nQ 24.46875 -20.796875 20.265625 -20.09375 \r\nQ 16.0625 -19.390625 12.109375 -17.921875 \r\nL 12.109375 -9.1875 \r\nQ 16.0625 -11.328125 19.921875 -12.34375 \r\nQ 23.78125 -13.375 27.78125 -13.375 \r\nQ 36.625 -13.375 41.015625 -8.765625 \r\nQ 45.40625 -4.15625 45.40625 5.171875 \r\nL 45.40625 9.625 \r\nQ 42.625 4.78125 38.28125 2.390625 \r\nQ 33.9375 0 27.875 0 \r\nQ 17.828125 0 11.671875 7.65625 \r\nQ 5.515625 15.328125 5.515625 27.984375 \r\nQ 5.515625 40.671875 11.671875 48.328125 \r\nQ 17.828125 56 27.875 56 \r\nQ 33.9375 56 38.28125 53.609375 \r\nQ 42.625 51.21875 45.40625 46.390625 \r\nL 45.40625 54.6875 \r\nL 54.390625 54.6875 \r\nz\r\n\" id=\"DejaVuSans-103\"/>\r\n      <path d=\"M 18.3125 70.21875 \r\nL 18.3125 54.6875 \r\nL 36.8125 54.6875 \r\nL 36.8125 47.703125 \r\nL 18.3125 47.703125 \r\nL 18.3125 18.015625 \r\nQ 18.3125 11.328125 20.140625 9.421875 \r\nQ 21.96875 7.515625 27.59375 7.515625 \r\nL 36.8125 7.515625 \r\nL 36.8125 0 \r\nL 27.59375 0 \r\nQ 17.1875 0 13.234375 3.875 \r\nQ 9.28125 7.765625 9.28125 18.015625 \r\nL 9.28125 47.703125 \r\nL 2.6875 47.703125 \r\nL 2.6875 54.6875 \r\nL 9.28125 54.6875 \r\nL 9.28125 70.21875 \r\nz\r\n\" id=\"DejaVuSans-116\"/>\r\n      <path d=\"M 54.890625 33.015625 \r\nL 54.890625 0 \r\nL 45.90625 0 \r\nL 45.90625 32.71875 \r\nQ 45.90625 40.484375 42.875 44.328125 \r\nQ 39.84375 48.1875 33.796875 48.1875 \r\nQ 26.515625 48.1875 22.3125 43.546875 \r\nQ 18.109375 38.921875 18.109375 30.90625 \r\nL 18.109375 0 \r\nL 9.078125 0 \r\nL 9.078125 75.984375 \r\nL 18.109375 75.984375 \r\nL 18.109375 46.1875 \r\nQ 21.34375 51.125 25.703125 53.5625 \r\nQ 30.078125 56 35.796875 56 \r\nQ 45.21875 56 50.046875 50.171875 \r\nQ 54.890625 44.34375 54.890625 33.015625 \r\nz\r\n\" id=\"DejaVuSans-104\"/>\r\n     </defs>\r\n     <g transform=\"translate(180.346094 255.068509)scale(0.1 -0.1)\">\r\n      <use xlink:href=\"#DejaVuSans-115\"/>\r\n      <use x=\"52.099609\" xlink:href=\"#DejaVuSans-101\"/>\r\n      <use x=\"113.623047\" xlink:href=\"#DejaVuSans-112\"/>\r\n      <use x=\"177.099609\" xlink:href=\"#DejaVuSans-97\"/>\r\n      <use x=\"238.378906\" xlink:href=\"#DejaVuSans-108\"/>\r\n      <use x=\"266.162109\" xlink:href=\"#DejaVuSans-32\"/>\r\n      <use x=\"297.949219\" xlink:href=\"#DejaVuSans-108\"/>\r\n      <use x=\"325.732422\" xlink:href=\"#DejaVuSans-101\"/>\r\n      <use x=\"387.255859\" xlink:href=\"#DejaVuSans-110\"/>\r\n      <use x=\"450.634766\" xlink:href=\"#DejaVuSans-103\"/>\r\n      <use x=\"514.111328\" xlink:href=\"#DejaVuSans-116\"/>\r\n      <use x=\"553.320312\" xlink:href=\"#DejaVuSans-104\"/>\r\n     </g>\r\n    </g>\r\n   </g>\r\n   <g id=\"matplotlib.axis_2\">\r\n    <g id=\"ytick_1\">\r\n     <g id=\"line2d_7\">\r\n      <defs>\r\n       <path d=\"M 0 0 \r\nL -3.5 0 \r\n\" id=\"mc9fd077e83\" style=\"stroke:#000000;stroke-width:0.8;\"/>\r\n      </defs>\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#mc9fd077e83\" y=\"216.90831\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_8\">\r\n      <!-- 2.0 -->\r\n      <defs>\r\n       <path d=\"M 19.1875 8.296875 \r\nL 53.609375 8.296875 \r\nL 53.609375 0 \r\nL 7.328125 0 \r\nL 7.328125 8.296875 \r\nQ 12.9375 14.109375 22.625 23.890625 \r\nQ 32.328125 33.6875 34.8125 36.53125 \r\nQ 39.546875 41.84375 41.421875 45.53125 \r\nQ 43.3125 49.21875 43.3125 52.78125 \r\nQ 43.3125 58.59375 39.234375 62.25 \r\nQ 35.15625 65.921875 28.609375 65.921875 \r\nQ 23.96875 65.921875 18.8125 64.3125 \r\nQ 13.671875 62.703125 7.8125 59.421875 \r\nL 7.8125 69.390625 \r\nQ 13.765625 71.78125 18.9375 73 \r\nQ 24.125 74.21875 28.421875 74.21875 \r\nQ 39.75 74.21875 46.484375 68.546875 \r\nQ 53.21875 62.890625 53.21875 53.421875 \r\nQ 53.21875 48.921875 51.53125 44.890625 \r\nQ 49.859375 40.875 45.40625 35.40625 \r\nQ 44.1875 33.984375 37.640625 27.21875 \r\nQ 31.109375 20.453125 19.1875 8.296875 \r\nz\r\n\" id=\"DejaVuSans-50\"/>\r\n      </defs>\r\n      <g transform=\"translate(20.878125 220.707528)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-50\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"ytick_2\">\r\n     <g id=\"line2d_8\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#mc9fd077e83\" y=\"175.726491\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_9\">\r\n      <!-- 2.5 -->\r\n      <g transform=\"translate(20.878125 179.52571)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-50\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"ytick_3\">\r\n     <g id=\"line2d_9\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#mc9fd077e83\" y=\"134.544673\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_10\">\r\n      <!-- 3.0 -->\r\n      <defs>\r\n       <path d=\"M 40.578125 39.3125 \r\nQ 47.65625 37.796875 51.625 33 \r\nQ 55.609375 28.21875 55.609375 21.1875 \r\nQ 55.609375 10.40625 48.1875 4.484375 \r\nQ 40.765625 -1.421875 27.09375 -1.421875 \r\nQ 22.515625 -1.421875 17.65625 -0.515625 \r\nQ 12.796875 0.390625 7.625 2.203125 \r\nL 7.625 11.71875 \r\nQ 11.71875 9.328125 16.59375 8.109375 \r\nQ 21.484375 6.890625 26.8125 6.890625 \r\nQ 36.078125 6.890625 40.9375 10.546875 \r\nQ 45.796875 14.203125 45.796875 21.1875 \r\nQ 45.796875 27.640625 41.28125 31.265625 \r\nQ 36.765625 34.90625 28.71875 34.90625 \r\nL 20.21875 34.90625 \r\nL 20.21875 43.015625 \r\nL 29.109375 43.015625 \r\nQ 36.375 43.015625 40.234375 45.921875 \r\nQ 44.09375 48.828125 44.09375 54.296875 \r\nQ 44.09375 59.90625 40.109375 62.90625 \r\nQ 36.140625 65.921875 28.71875 65.921875 \r\nQ 24.65625 65.921875 20.015625 65.03125 \r\nQ 15.375 64.15625 9.8125 62.3125 \r\nL 9.8125 71.09375 \r\nQ 15.4375 72.65625 20.34375 73.4375 \r\nQ 25.25 74.21875 29.59375 74.21875 \r\nQ 40.828125 74.21875 47.359375 69.109375 \r\nQ 53.90625 64.015625 53.90625 55.328125 \r\nQ 53.90625 49.265625 50.4375 45.09375 \r\nQ 46.96875 40.921875 40.578125 39.3125 \r\nz\r\n\" id=\"DejaVuSans-51\"/>\r\n      </defs>\r\n      <g transform=\"translate(20.878125 138.343892)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-51\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"ytick_4\">\r\n     <g id=\"line2d_10\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#mc9fd077e83\" y=\"93.362855\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_11\">\r\n      <!-- 3.5 -->\r\n      <g transform=\"translate(20.878125 97.162074)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-51\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"ytick_5\">\r\n     <g id=\"line2d_11\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#mc9fd077e83\" y=\"52.181037\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_12\">\r\n      <!-- 4.0 -->\r\n      <g transform=\"translate(20.878125 55.980256)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-52\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-48\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"ytick_6\">\r\n     <g id=\"line2d_12\">\r\n      <g>\r\n       <use style=\"stroke:#000000;stroke-width:0.8;\" x=\"43.78125\" xlink:href=\"#mc9fd077e83\" y=\"10.999219\"/>\r\n      </g>\r\n     </g>\r\n     <g id=\"text_13\">\r\n      <!-- 4.5 -->\r\n      <g transform=\"translate(20.878125 14.798437)scale(0.1 -0.1)\">\r\n       <use xlink:href=\"#DejaVuSans-52\"/>\r\n       <use x=\"63.623047\" xlink:href=\"#DejaVuSans-46\"/>\r\n       <use x=\"95.410156\" xlink:href=\"#DejaVuSans-53\"/>\r\n      </g>\r\n     </g>\r\n    </g>\r\n    <g id=\"text_14\">\r\n     <!-- sepal width -->\r\n     <defs>\r\n      <path d=\"M 4.203125 54.6875 \r\nL 13.1875 54.6875 \r\nL 24.421875 12.015625 \r\nL 35.59375 54.6875 \r\nL 46.1875 54.6875 \r\nL 57.421875 12.015625 \r\nL 68.609375 54.6875 \r\nL 77.59375 54.6875 \r\nL 63.28125 0 \r\nL 52.6875 0 \r\nL 40.921875 44.828125 \r\nL 29.109375 0 \r\nL 18.5 0 \r\nz\r\n\" id=\"DejaVuSans-119\"/>\r\n      <path d=\"M 9.421875 54.6875 \r\nL 18.40625 54.6875 \r\nL 18.40625 0 \r\nL 9.421875 0 \r\nz\r\nM 9.421875 75.984375 \r\nL 18.40625 75.984375 \r\nL 18.40625 64.59375 \r\nL 9.421875 64.59375 \r\nz\r\n\" id=\"DejaVuSans-105\"/>\r\n      <path d=\"M 45.40625 46.390625 \r\nL 45.40625 75.984375 \r\nL 54.390625 75.984375 \r\nL 54.390625 0 \r\nL 45.40625 0 \r\nL 45.40625 8.203125 \r\nQ 42.578125 3.328125 38.25 0.953125 \r\nQ 33.9375 -1.421875 27.875 -1.421875 \r\nQ 17.96875 -1.421875 11.734375 6.484375 \r\nQ 5.515625 14.40625 5.515625 27.296875 \r\nQ 5.515625 40.1875 11.734375 48.09375 \r\nQ 17.96875 56 27.875 56 \r\nQ 33.9375 56 38.25 53.625 \r\nQ 42.578125 51.265625 45.40625 46.390625 \r\nz\r\nM 14.796875 27.296875 \r\nQ 14.796875 17.390625 18.875 11.75 \r\nQ 22.953125 6.109375 30.078125 6.109375 \r\nQ 37.203125 6.109375 41.296875 11.75 \r\nQ 45.40625 17.390625 45.40625 27.296875 \r\nQ 45.40625 37.203125 41.296875 42.84375 \r\nQ 37.203125 48.484375 30.078125 48.484375 \r\nQ 22.953125 48.484375 18.875 42.84375 \r\nQ 14.796875 37.203125 14.796875 27.296875 \r\nz\r\n\" id=\"DejaVuSans-100\"/>\r\n     </defs>\r\n     <g transform=\"translate(14.798438 146.750852)rotate(-90)scale(0.1 -0.1)\">\r\n      <use xlink:href=\"#DejaVuSans-115\"/>\r\n      <use x=\"52.099609\" xlink:href=\"#DejaVuSans-101\"/>\r\n      <use x=\"113.623047\" xlink:href=\"#DejaVuSans-112\"/>\r\n      <use x=\"177.099609\" xlink:href=\"#DejaVuSans-97\"/>\r\n      <use x=\"238.378906\" xlink:href=\"#DejaVuSans-108\"/>\r\n      <use x=\"266.162109\" xlink:href=\"#DejaVuSans-32\"/>\r\n      <use x=\"297.949219\" xlink:href=\"#DejaVuSans-119\"/>\r\n      <use x=\"379.736328\" xlink:href=\"#DejaVuSans-105\"/>\r\n      <use x=\"407.519531\" xlink:href=\"#DejaVuSans-100\"/>\r\n      <use x=\"470.996094\" xlink:href=\"#DejaVuSans-116\"/>\r\n      <use x=\"510.205078\" xlink:href=\"#DejaVuSans-104\"/>\r\n     </g>\r\n    </g>\r\n   </g>\r\n   <g id=\"line2d_13\">\r\n    <defs>\r\n     <path d=\"M 0 3 \r\nC 0.795609 3 1.55874 2.683901 2.12132 2.12132 \r\nC 2.683901 1.55874 3 0.795609 3 0 \r\nC 3 -0.795609 2.683901 -1.55874 2.12132 -2.12132 \r\nC 1.55874 -2.683901 0.795609 -3 0 -3 \r\nC -0.795609 -3 -1.55874 -2.683901 -2.12132 -2.12132 \r\nC -2.683901 -1.55874 -3 -0.795609 -3 0 \r\nC -3 0.795609 -2.683901 1.55874 -2.12132 2.12132 \r\nC -1.55874 2.683901 -0.795609 3 0 3 \r\nz\r\n\" id=\"m5450b4bb8b\" style=\"stroke:#0000ff;\"/>\r\n    </defs>\r\n    <g clip-path=\"url(#pc48046d1f2)\">\r\n     <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"250.635795\" xlink:href=\"#m5450b4bb8b\" y=\"134.544673\"/>\r\n    </g>\r\n   </g>\r\n   <g id=\"patch_3\">\r\n    <path d=\"M 43.78125 226.791946 \r\nL 43.78125 9.351946 \r\n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n   </g>\r\n   <g id=\"patch_4\">\r\n    <path d=\"M 378.58125 226.791946 \r\nL 378.58125 9.351946 \r\n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n   </g>\r\n   <g id=\"patch_5\">\r\n    <path d=\"M 43.78125 226.791946 \r\nL 378.58125 226.791946 \r\n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n   </g>\r\n   <g id=\"patch_6\">\r\n    <path d=\"M 43.78125 9.351946 \r\nL 378.58125 9.351946 \r\n\" style=\"fill:none;stroke:#000000;stroke-linecap:square;stroke-linejoin:miter;stroke-width:0.8;\"/>\r\n   </g>\r\n   <g id=\"legend_1\">\r\n    <g id=\"patch_7\">\r\n     <path d=\"M 289.875 61.664446 \r\nL 371.58125 61.664446 \r\nQ 373.58125 61.664446 373.58125 59.664446 \r\nL 373.58125 16.351946 \r\nQ 373.58125 14.351946 371.58125 14.351946 \r\nL 289.875 14.351946 \r\nQ 287.875 14.351946 287.875 16.351946 \r\nL 287.875 59.664446 \r\nQ 287.875 61.664446 289.875 61.664446 \r\nz\r\n\" style=\"fill:#ffffff;opacity:0.8;stroke:#cccccc;stroke-linejoin:miter;\"/>\r\n    </g>\r\n    <g id=\"line2d_14\"/>\r\n    <g id=\"line2d_15\">\r\n     <g>\r\n      <use style=\"fill:#0000ff;stroke:#0000ff;\" x=\"301.875\" xlink:href=\"#m5450b4bb8b\" y=\"22.450384\"/>\r\n     </g>\r\n    </g>\r\n    <g id=\"text_15\">\r\n     <!-- test_point -->\r\n     <defs>\r\n      <path d=\"M 50.984375 -16.609375 \r\nL 50.984375 -23.578125 \r\nL -0.984375 -23.578125 \r\nL -0.984375 -16.609375 \r\nz\r\n\" id=\"DejaVuSans-95\"/>\r\n      <path d=\"M 30.609375 48.390625 \r\nQ 23.390625 48.390625 19.1875 42.75 \r\nQ 14.984375 37.109375 14.984375 27.296875 \r\nQ 14.984375 17.484375 19.15625 11.84375 \r\nQ 23.34375 6.203125 30.609375 6.203125 \r\nQ 37.796875 6.203125 41.984375 11.859375 \r\nQ 46.1875 17.53125 46.1875 27.296875 \r\nQ 46.1875 37.015625 41.984375 42.703125 \r\nQ 37.796875 48.390625 30.609375 48.390625 \r\nz\r\nM 30.609375 56 \r\nQ 42.328125 56 49.015625 48.375 \r\nQ 55.71875 40.765625 55.71875 27.296875 \r\nQ 55.71875 13.875 49.015625 6.21875 \r\nQ 42.328125 -1.421875 30.609375 -1.421875 \r\nQ 18.84375 -1.421875 12.171875 6.21875 \r\nQ 5.515625 13.875 5.515625 27.296875 \r\nQ 5.515625 40.765625 12.171875 48.375 \r\nQ 18.84375 56 30.609375 56 \r\nz\r\n\" id=\"DejaVuSans-111\"/>\r\n     </defs>\r\n     <g transform=\"translate(319.875 25.950384)scale(0.1 -0.1)\">\r\n      <use xlink:href=\"#DejaVuSans-116\"/>\r\n      <use x=\"39.208984\" xlink:href=\"#DejaVuSans-101\"/>\r\n      <use x=\"100.732422\" xlink:href=\"#DejaVuSans-115\"/>\r\n      <use x=\"152.832031\" xlink:href=\"#DejaVuSans-116\"/>\r\n      <use x=\"192.041016\" xlink:href=\"#DejaVuSans-95\"/>\r\n      <use x=\"242.041016\" xlink:href=\"#DejaVuSans-112\"/>\r\n      <use x=\"305.517578\" xlink:href=\"#DejaVuSans-111\"/>\r\n      <use x=\"366.699219\" xlink:href=\"#DejaVuSans-105\"/>\r\n      <use x=\"394.482422\" xlink:href=\"#DejaVuSans-110\"/>\r\n      <use x=\"457.861328\" xlink:href=\"#DejaVuSans-116\"/>\r\n     </g>\r\n    </g>\r\n    <g id=\"PathCollection_3\">\r\n     <g>\r\n      <use style=\"fill:#1f77b4;stroke:#1f77b4;\" x=\"301.875\" xlink:href=\"#me5904ac1a2\" y=\"38.281634\"/>\r\n     </g>\r\n    </g>\r\n    <g id=\"text_16\">\r\n     <!-- 0 -->\r\n     <g transform=\"translate(319.875 40.906634)scale(0.1 -0.1)\">\r\n      <use xlink:href=\"#DejaVuSans-48\"/>\r\n     </g>\r\n    </g>\r\n    <g id=\"PathCollection_4\">\r\n     <g>\r\n      <use style=\"fill:#ff7f0e;stroke:#ff7f0e;\" x=\"301.875\" xlink:href=\"#m5fb07bbd4d\" y=\"52.959759\"/>\r\n     </g>\r\n    </g>\r\n    <g id=\"text_17\">\r\n     <!-- 1 -->\r\n     <defs>\r\n      <path d=\"M 12.40625 8.296875 \r\nL 28.515625 8.296875 \r\nL 28.515625 63.921875 \r\nL 10.984375 60.40625 \r\nL 10.984375 69.390625 \r\nL 28.421875 72.90625 \r\nL 38.28125 72.90625 \r\nL 38.28125 8.296875 \r\nL 54.390625 8.296875 \r\nL 54.390625 0 \r\nL 12.40625 0 \r\nz\r\n\" id=\"DejaVuSans-49\"/>\r\n     </defs>\r\n     <g transform=\"translate(319.875 55.584759)scale(0.1 -0.1)\">\r\n      <use xlink:href=\"#DejaVuSans-49\"/>\r\n     </g>\r\n    </g>\r\n   </g>\r\n  </g>\r\n </g>\r\n <defs>\r\n  <clipPath id=\"pc48046d1f2\">\r\n   <rect height=\"217.44\" width=\"334.8\" x=\"43.78125\" y=\"9.351946\"/>\r\n  </clipPath>\r\n </defs>\r\n</svg>\r\n",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEJCAYAAACZjSCSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de5xVdb3/8dfHcRJQghI6CQNOXn78SkAu4y285KXwgklkpklG+jiEx5OWSUV6NC2PevSXHvSXhmWZIkoewjJESyUtjxg3QUVD88LFXxIFgqAHx8/vj7UGhs2emb1m9nfvtfZ6Px+PeezZa6+95vNdW/eHtdbns77m7oiISH7tUu0ARESkupQIRERyTolARCTnlAhERHJOiUBEJOeUCEREci54IjCzOjNbbGb3F3ntE2a2wcyWxD+Xho5HRER2tGsF/sYFwHLg/W28/ri7j6lAHCIiUkTQRGBmDcBJwJXAheXYZp8+fbyxsbEcmxIRyY2FCxf+zd37Fnst9BHBDcA3gZ7trHOYmT0NrAEucvdn29tgY2MjCxYsKGOIIiK1z8xebeu1YNcIzGwM8Ia7L2xntUXA3u5+IHAjMLuNbU00swVmtmDt2rUBohURya+QF4tHAZ82s1eAu4FjzOzO1iu4+5vuvin+fQ5Qb2Z9Cjfk7tPcvcndm/r2LXpkIyIinRQsEbj7FHdvcPdG4HTgEXcf33odM/uwmVn8+8FxPOtCxSQiIjurRNXQDsxsEoC73wKcCpxrZu8CW4DTXbdDFcmVrVu3smrVKt5+++1qh1ITunXrRkNDA/X19SW/x7L2vdvU1OS6WCxSO15++WV69uzJnnvuSXyCQDrJ3Vm3bh0bN27kIx/5yA6vmdlCd28q9r6KHxGI5MXsxau59sEXWLN+C/16d2fy6EGMHd6/2mGlzttvv01jY6OSQBmYGXvuuSdJi2qUCEQCmL14NVNmLWPL1mYAVq/fwpRZywCUDIpQEiifzuxL3WtIJIBrH3xhWxJosWVrM9c++EKVIhJpmxKBSABr1m9JtFyqZ/369fzwhz/s1HtvuOEGNm/eXNZ4Lr30Un73u9+1u868efN44oknyvY3lQhEAujXu3ui5VK66dOhsRF22SV6nD69a9tLWyK44oorOO6449pdR4lAJAMmjx5E9/q6HZZ1r69j8uhBVYqoNkyfDhMnwquvgnv0OHFi15LBt7/9bV566SWGDRvG5MmTufbaaznooIMYOnQol112GQBvvfUWJ510EgceeCCDBw/mnnvuYerUqaxZs4ajjz6ao48+us3t77HHHnzjG99gxIgRHHvssdsu5C5ZsoRDDz2UoUOH8pnPfIZ//OMfAEyYMIF7770XiG6pc9lllzFixAiGDBnC888/zyuvvMItt9zC9ddfz7Bhw3j88cc7P/iYEoFIAGOH9+eqcUPo37s7BvTv3Z2rxg3RheIuuvhiKPwH+ObN0fLOuvrqq9l3331ZsmQJn/zkJ1mxYgVPPfUUS5YsYeHChTz22GPMnTuXfv368fTTT/PMM89w/PHHc/7559OvXz8effRRHn300Ta3/9ZbbzFixAgWLVrEUUcdxeWXXw7AWWedxTXXXMPSpUsZMmTItuWF+vTpw6JFizj33HO57rrraGxsZNKkSXz9619nyZIlHHHEEZ0ffExVQyKBjB3eX1/8Zfbaa8mWJ/XQQw/x0EMPMXz4cAA2bdrEihUrOOKII7jooov41re+xZgxYxJ9+e6yyy58/vOfB2D8+PGMGzeODRs2sH79eo466igAvvSlL/G5z32u6PvHjRsHwMiRI5k1a1ZXhtcmJQIRyYyBA6PTQcWWl4O7M2XKFL7yla/s9NrChQuZM2cOU6ZM4VOf+hSXXtq5ebSSlnfutttuANTV1fHuu+926m92RKeGRCQzrrwSevTYcVmPHtHyzurZsycbN24EYPTo0dx2221s2rQJgNWrV/PGG2+wZs0aevTowfjx47noootYtGjRTu9ty3vvvbftnP9dd93F4YcfTq9evfjABz6w7fz+HXfcse3oIGnM5aAjAhHJjDPPjB4vvjg6HTRwYJQEWpZ3xp577smoUaMYPHgwJ5xwAl/4whc47LDDgOhC75133smLL77I5MmT2WWXXaivr+fmm28GYOLEiZxwwgnstddebV4n2H333Xn22WcZOXIkvXr14p577gHg9ttvZ9KkSWzevJl99tmHn/70pyXHfPLJJ3Pqqady3333ceONN3b5OoHuNSQiVbV8+XI++tGPVjuMYPbYY49tRxiVUmyftnevIZ0aEhHJOZ0aEhEpg0MOOYR33nlnh2V33HFHxY8GOkOJQESkDObPn1/tEDpNp4ZERHJOiUBEJOd0akhyTxPISN7piEByrWUCmdXrt+Bsn0Bm9uLV1Q5NKmju3LkMGjSI/fbbj6uvvrra4VScEoHkmiaQkebmZs477zweeOABnnvuOWbMmMFzzz1X7bAqSolAck0TyMhTTz3Ffvvtxz777MP73vc+Tj/9dO67775qh1VRukYgudavd3dWF/nS1wQy6VXuazqrV69mwIAB2543NDRkuhS0M3REILmmCWSyJcQ1nWK32enMBPBZpkQguaYJZLIlxDWdhoYGVq5cue35qlWr6NevX6e3l0U6NSS5pwlksiPENZ2DDjqIFStW8PLLL9O/f3/uvvtu7rrrrk5vL4uUCKRqVL8vSYW4prPrrrty0003MXr0aJqbmzn77LM54IADuhJm5igRSFW0nOttOcxvOdcLKBlImyaPHrTDfzdQnms6J554IieeeGJXw8ssXSOQqlD9vnSGrumEoSMCqQrV70tn6ZpO+emIQKqirXO6qt8XqTwlAqkK1e+LpIdODUlVtBzaq2pIpPqCJwIzqwMWAKvdfUzBawb8J3AisBmY4O6LQsck6aBzvSLpUIlTQxcAy9t47QRg//hnInBzBeIRSaXZi1cz6upH+Mi3f8Ooqx/RrbAr6Oyzz+ZDH/oQgwcPrnYoVRE0EZhZA3AS8OM2VjkF+LlHngR6m9leIWMSSSPNi1BdEyZMYO7cudUOo2pCHxHcAHwTeK+N1/sDK1s9XxUvE8kV9VVU15FHHskHP/jBaodRNcESgZmNAd5w94XtrVZk2U63AjSziWa2wMwWrF27tmwxiqSF+ioSWDoTrh8M3+0dPS6dWe2IMi/kEcEo4NNm9gpwN3CMmd1ZsM4qYECr5w3AmsINufs0d29y96a+ffuGilekatRXUaKlM+HX58OGlYBHj78+X8mgi4IlAnef4u4N7t4InA484u7jC1b7FXCWRQ4FNrj766FiEkkr9VWU6OErYGvBUdLWLdFy6bSK9xGY2SQAd78FmENUOvoiUfnolysdj0gaqK+iRBtWJVsuJalIInD3ecC8+PdbWi134LxKxCCSduqrKEGvhvi0UJHlXXDGGWcwb948/va3v9HQ0MDll1/OOeec06VtZok6i6XmXDJ7GTPmr6TZnTozzjhkAN8fO6TaYUk5HHtpdE2g9emh+u7R8i6YMWNGFwPLNt1rSGrKJbOXceeTr9Ecz0Pb7M6dT77GJbOXVTkyKYuhp8HJU6HXAMCix5OnRsul03REIDVlxvwipw3i5ToqqBFDT9MXf5npiEBqSsuRQKnLRUSJQGpMnRXrUWx7uaSDK1GXTWf2pRKB1JQzDhmQaLlUX7du3Vi3bp2SQRm4O+vWraNbt26J3qdrBFJTWq4DqGooOxoaGli1ahW6fUx5dOvWjYaGZOW0lrUs3NTU5AsWLKh2GCIimWJmC929qdhrOjUkIpJzOjUkZXXmrf/NH1/6+7bno/b9INP/+bAqRlQ9sxev1i0jJBN0RCBlU5gEAP740t8589b/rlJE1aOJZiRLlAikbAqTQEfLa5kmmpEsUSIQCUATzUiWKBGIBKCJZiRLlAikbEbtW3zO17aW1zJNNCNZokQgZTP9nw/b6Us/r1VDY4f356pxQ+jfuzsG9O/dnavGDVHVkKSSGspERHKgvYYy9RFIWYWqnU+yXdXviySjRCBl01I731I22VI7D3TpizjJdkPFIFLLdI1AyiZU7XyS7ap+XyQ5JQIpm1C180m2q/p9keSUCKRsQtXOJ9mu6vdFklMikLIJVTufZLuq3xdJTheLpWxaLsaWu2InyXZDxSBSy9RHICKSA+ojSIm01LerJl9EWlMiqJC01LerJl9ECulicYWkpb5dNfkiUkiJoELSUt+umnwRKaREUCFpqW9XTb6IFFIiqJC01LerJl9ECulicYWkpb5dNfkiUkh9BCIiOVCVPgIz6wY8BuwW/5173f2ygnU+AdwHvBwvmuXuV4SKSZK7ZPYyZsxfSbM7dWacccgAvj92SFnWT0uPQlriEKmWDhOBme0GfBZobL1+CV/Y7wDHuPsmM6sH/mBmD7j7kwXrPe7uY5KFLZVwyexl3Pnka9ueN7tve17syz3J+mnpUUhLHCLVVMrF4vuAU4B3gbda/bTLI5vip/XxT7bOQ+XcjPkrgy1PS49CWuIQqaZSTg01uPvxndm4mdUBC4H9gP/r7vOLrHaYmT0NrAEucvdni2xnIjARYODAgZ0JRTqhuY3rR+VYnpYehbTEIVJNpRwRPGFmbZ8Uboe7N7v7MKABONjMBhessgjY290PBG4EZrexnWnu3uTuTX379u1MKNIJdWbBlqelRyEtcYhUU5uJwMyWmdlS4HBgkZm9YGZLWy0vmbuvB+YBxxcsf7Pl9JG7zwHqzaxP0kFIGGccMiDY8rT0KKQlDpFqau/UUJcu4JpZX2Cru683s+7AccA1Bet8GPiru7uZHUyUmNZ15e9K+bRc4C21CijJ+mnpUUhLHCLV1GEfgZnd4e5f7GhZkfcNBW4H6oi+4Ge6+xVmNgnA3W8xs38FziW6EL0FuNDdn2hvu+ojEBFJrqt9BAcUbKwOGNnRm9x9KTC8yPJbWv1+E3BTCTGIiEggbSYCM5sCfAfobmZvtiwG/geYVoHYak7IxqWkjV+htpuGSW9C7YvMWjoTHr4CNqyCXg1w7KUw9LRqRyUp0mYicPergKvM7Cp3n1LBmGpSyMalpI1fobabhklvQu2LzFo6E359PmyNy2E3rIyeg5KBbNNe1dAIMxsB/KLl99Y/FYyxJoRsXEra4BVqu2mY9CbUvsish6/YngRabN0SLReJtXeN4P/Ej92AJuBpolNDQ4H5RGWlUqKQjUtJG7xCbTcNk96E2heZtWFVsuWSS20eEbj70e5+NPAqMCJu6BpJdAH4xUoFWCtCNi4lbfAKtd00THoTal9kVq+GZMsll0rpLP7f7r6s5Ym7PwMMCxdSbQrZuJS0wSvUdtMw6U2ofZFZx14K9QXJtb57tFwkVkr56HIz+zFwJ9FN48YDy4NGVYNCNi4lbfwKtd00THoTal9kVssFYVUNSTtKaSjrRtT0dWS86DHgZnd/O3BsRamhTEQkuS41lMVf+NfHP5IzSWv9NcmLtEn9DKnVXkPZTHc/zcyWUWQeAXcfGjQyqbqktf6a5EXapH6GVGvvYvEF8eMY4OQiP1Ljktb6a5IXaZP6GVKtvc7i1+NfjyWaTnJFZUKStEha669JXqRN6mdItVLKRxuBH5nZS2Y208y+amYqH82BpLX+muRF2qR+hlTrMBG4+6XufgwwGPgDMJlo+kmpcUlr/TXJi7RJ/Qyp1mHVkJldAowC9gAWAxcBjweOS1Igaa2/JnmRNqmfIdVK6SNYRDRxzG+A3wNPVquHANRHICLSGV3tIxhhZj2JbjL3SeBWM/uru9fsTedC1cIn3W4a7quvvoCUqvWa/FofX1KB90cpp4YGA0cARxHdhXQlNXxqKFQtfNLtpuG++uoLSKlar8mv9fElVYH9UUrV0DVAT2Aq8NH4rqQ1e4UnVC180u2m4b766gtIqVqvya/18SVVgf1Ryqmhk8r21zIgVC180u2m4b766gtIqVqvya/18SVVgf1RyhFBroSqhU+63TTcV199ASlV6zX5tT6+pCqwP5QICoSqhU+63TTcV199ASlV6zX5tT6+pCqwP0qZjyBXQtXCJ91uGu6rr76AlKr1mvxaH19SFdgfbfYRmNmvKXLX0Rbu/umyRZGA+ghERJLrbB/BdYHiya2QNflJtp2G/gSRTLj/Qlj4M/BmsDoYOQHG/KA8205Rr0R7dx/9fSUDqXUha/KTbDsN/QkimXD/hbDgJ9ufe/P2511NBinrlejwYrGZ7W9m95rZc2b2l5afSgRXS0LW5CfZdhr6E0QyYeHPki1PImW9EqVUDf0UuJnofkNHAz8H7ggZVC0KWZOfZNtp6E8QyQRvTrY8iZT1SpSSCLq7+8NEF5ZfdffvAseEDav2hKzJT7LtNPQniGSC1SVbnkTKeiVKSQRvm9kuwAoz+1cz+wzwocBx1ZyQNflJtp2G/gSRTBg5IdnyJFLWK1FKH8HXgB7A+cD3iI4GvhQyqFoUsiY/ybbT0J8gkgktF4RDVA2lrFeiw/kItq1o9n7A3X1j2JDapz4CEZHk2usjKKVqqMnMlgFLgWVm9rSZjSzhfd3M7Kl4/WfN7PIi65iZTTWzF81sqZmNKGVAIiJSPqWcGroN+Bd3fxzAzA4nqiQa2sH73gGOcfdNZlYP/MHMHnD3J1utcwKwf/xzCFF10iEJx9ChpI1cWZyMJUmTWJLxZXFfBG3USdJgFDKOBNuePh0uvhheew0GDoQrr4Qzz+z6djMryRjzsD8oLRFsbEkCAO7+BzPr8PSQR+ecNsVP6+OfwvNQpwA/j9d90sx6m9le7v56aeF3LGkjVxYnY0nSJJZkfFncF0EbdZI0GIWMI8G2p0+HiRNh8+bo+auvRs+hSDJIWZNTEEnGmIf9ESulaugpM/uRmX3CzI4ysx8C88xsREencsyszsyWAG8Av3X3+QWr9Cea8azFqnhZ2SRt5MriZCxJmsSSjC+L+yJoo06SBqOQcSTY9sUXb08CLTZvjpZ3ZbuZlWSMedgfsVKOCIbFj5cVLP840b/w2+wpcPdmYJiZ9QZ+aWaD3f2ZVqsUK17f6eq1mU0EJgIMHDiwhJC3S9rIlcXJWJI0iSUZXxb3RdBGnSQNRiHjSLDt114rsl5by1PW5BREkjHmYX/EOjwiiKembOunpMYyd18PzAOOL3hpFdC6gL0BWFPk/dPcvcndm/r27VvKn9wmaSNXFidjSdIklmR8WdwXQRt1kjQYhYwjwbbb+ndT0eUpa3IKIskY87A/YqVUDf2Tmf3EzB6In3/MzM4p4X194yMBzKw7cBzwfMFqvwLOiquHDgU2lPP6ACRv5MriZCxJmsSSjC+L+yJoo06SBqOQcSTY9pVXQo8eOy7r0SNa3pXtZlaSMeZhf8RKuUbwM+BBoF/8/M9ETWYd2Qt41MyWAn8iukZwv5lNMrNJ8TpzgL8ALwK3Av+SIPaSjB3en6vGDaF/7+4Y0L93d64aN6TNi51J10+D748dwvhDB247AqgzY/yhA4tWDSUZXxb3BUNPg5OnQq8BgEWPJ08tz8W9MT+ApnO2HwFYXfS8WNVQyDgSbPvMM2HaNNh7bzCLHqdNa6NqKGTMaZFkjHnYH7EOG8rM7E/ufpCZLXb34fGyJe4+rN03BqKGMhGR5Do7MU2Lt8xsT+KLuC2ncMoYX+pksnZeKiOLNeghYw41xpT0YORFKYngQqJz+fua2R+BvsCpQaOqokzWzktlZLEGPWTMocaYkh6MPCmlamgRcBRRuehXgAPcfWnowKolk7XzUhlZrEEPGXOoMaakByNPSqka+hzRnATPAmOBe2r5nkCZrJ2XyshiDXrImEONMSU9GHlSStXQv7n7xvgeQ6OB24nuCVSTMlk7L5WRxRr0kDGHGmNKejDypJRE0HKe5CTgZne/D3hfuJCqK5O181IZWaxBDxlzqDGmpAcjT0pJBKvN7EfAacAcM9utxPdlUiZr56UysliDHjLmUGNMSQ9GnpTSR9CD6NYQy9x9hZntBQxx94cqEWAh9RGIiCTXpT4Cd98MzGr1/HWgrLeBEKlJSeYuSIssxpyWvoC0xNEJpfQRiEhSSeYuSIssxpyWvoC0xNFJNXuuX6SqksxdkBZZjDktfQFpiaOTlAhEQkgyd0FaZDHmtPQFpCWOTlIiEAkhydwFaZHFmNPSF5CWODpJiUAkhCRzF6RFFmNOS19AWuLoJCUCkRCSzF2QFlmMOS19AWmJo5M67CNIG/URiIgk19X5CETCyGLddciYQ9XwZ3E/S0UpEUh1ZLHuOmTMoWr4s7ifpeJ0jUCqI4t11yFjDlXDn8X9LBWnRCDVkcW665Axh6rhz+J+lopTIpDqyGLddciYQ9XwZ3E/S8UpEUh1ZLHuOmTMoWr4s7ifpeKUCKQ6slh3HTLmUDX8WdzPUnHqIxARyYH2+gh0RCCydCZcPxi+2zt6XDqzOtsNFYdIB9RHIPkWqs4+6XZV7y9VpCMCybdQdfZJt6t6f6kiJQLJt1B19km3q3p/qSIlAsm3UHX2Sberen+pIiUCybdQdfZJt6t6f6kiJQLJt1B19km3q3p/qSL1EYiI5EBV+gjMbICZPWpmy83sWTO7oMg6nzCzDWa2JP7RcbCISIWF7CN4F/iGuy8ys57AQjP7rbs/V7De4+4+JmAcUklZnAQlScxZHF9aaN+lVrBE4O6vA6/Hv280s+VAf6AwEUityGJTVJKYszi+tNC+S7WKXCw2s0ZgODC/yMuHmdnTZvaAmR1QiXgkkCw2RSWJOYvjSwvtu1QLfosJM9sD+C/ga+7+ZsHLi4C93X2TmZ0IzAb2L7KNicBEgIEDBwaOWDoti01RSWLO4vjSQvsu1YIeEZhZPVESmO7uswpfd/c33X1T/PscoN7M+hRZb5q7N7l7U9++fUOGLF2RxaaoJDFncXxpoX2XaiGrhgz4CbDc3YveVN3MPhyvh5kdHMezLlRMElgWm6KSxJzF8aWF9l2qhTw1NAr4IrDMzJbEy74DDARw91uAU4FzzexdYAtwumetsUG2a7nol6XKkCQxZ3F8aaF9l2pqKBMRyYH2Gso0H0EeqZ57R/dfCAt/Bt4cTRE5ckLXp4gUyRAlgrxRPfeO7r8QFvxk+3Nv3v5cyUByQjedyxvVc+9o4c+SLRepQUoEeaN67h15c7LlIjVIiSBvVM+9I6tLtlykBikR5I3quXc0ckKy5SI1SIkgbzQByo7G/ACaztl+BGB10XNdKJYcUR+BiEgOqI8goNmLV3Ptgy+wZv0W+vXuzuTRgxg7vH+1wyqfPPQc5GGMaaD9nFpKBF0we/FqpsxaxpatUYXJ6vVbmDJrGUBtJIM89BzkYYxpoP2carpG0AXXPvjCtiTQYsvWZq598IUqRVRmeeg5yMMY00D7OdWUCLpgzfotiZZnTh56DvIwxjTQfk41JYIu6Ne7e6LlmZOHnoM8jDENtJ9TTYmgCyaPHkT3+h0bj7rX1zF59KAqRVRmeeg5yMMY00D7OdV0sbgLWi4I12zVUB7uIZ+HMaaB9nOqqY9ARCQH2usj0KkhkVq3dCZcPxi+2zt6XDozG9uWitGpIZFaFrJ+X70BNUNHBCK1LGT9vnoDaoYSgUgtC1m/r96AmqFEIFLLQtbvqzegZigRiNSykPX76g2oGUoEIrUs5PwTmtuiZqiPQEQkB9RHICIibVIiEBHJOSUCEZGcUyIQEck5JQIRkZxTIhARyTklAhGRnFMiEBHJuWCJwMwGmNmjZrbczJ41swuKrGNmNtXMXjSzpWY2IlQ8IiJSXMgjgneBb7j7R4FDgfPM7GMF65wA7B//TARuDhiPdIYmHhGpecESgbu/7u6L4t83AsuBwsl8TwF+7pEngd5mtleomCShlolHNqwEfPvEI0oGIjWlItcIzKwRGA7ML3ipP7Cy1fNV7JwspFo08YhILgRPBGa2B/BfwNfc/c3Cl4u8Zae74JnZRDNbYGYL1q5dGyJMKUYTj4jkQtBEYGb1RElgurvPKrLKKmBAq+cNwJrCldx9mrs3uXtT3759wwQrO9PEIyK5ELJqyICfAMvd/QdtrPYr4Ky4euhQYIO7vx4qJklIE4+I5MKuAbc9CvgisMzMlsTLvgMMBHD3W4A5wInAi8Bm4MsB45GkWiYYefiK6HRQr4YoCWjiEZGaoolpRERyQBPTiIhIm5QIRERyTolARCTnlAhERHJOiUBEJOcyVzVkZmuBV6sdRxF9gL9VO4iAan18UPtj1Piyrytj3Nvdi3bkZi4RpJWZLWirNKsW1Pr4oPbHqPFlX6gx6tSQiEjOKRGIiOScEkH5TKt2AIHV+vig9seo8WVfkDHqGoGISM7piEBEJOeUCBIyszozW2xm9xd57RNmtsHMlsQ/mbtfs5m9YmbL4vh3urtffMvwqWb2opktNbMR1YizK0oYY6Y/RzPrbWb3mtnzZrbczA4reD3Tn2EJ48v65zeoVexLzOxNM/tawTpl/QxD3oa6Vl1ANP/y+9t4/XF3H1PBeEI42t3bqlU+Adg//jkEuDl+zJr2xgjZ/hz/E5jr7qea2fuAHgWvZ/0z7Gh8kOHPz91fAIZB9A9PYDXwy4LVyvoZ6oggATNrAE4CflztWKroFODnHnkS6G1me1U7KImY2fuBI4kmhcLd/8fd1xesltnPsMTx1ZJjgZfcvbCJtqyfoRJBMjcA3wTea2edw8zsaTN7wMwOqFBc5eTAQ2a20MwmFnm9P7Cy1fNV8bIs6WiMkN3PcR9gLfDT+BTmj81s94J1svwZljI+yO7nV+h0YEaR5WX9DJUISmRmY4A33H1hO6stImrjPhC4EZhdkeDKa5S7jyA69DzPzI4seN2KvCdrpWcdjTHLn+OuwAjgZncfDrwFfLtgnSx/hqWML8uf3zbxaa9PA78o9nKRZZ3+DJUISjcK+LSZvQLcDRxjZne2XsHd33T3TfHvc4B6M+tT8Ui7wN3XxI9vEJ2XPLhglVXAgFbPG4A1lYmuPDoaY8Y/x1XAKnefHz+/l+iLs3CdrH6GHY4v459faycAi9z9r0VeK+tnqERQInef4u4N7t5IdLj2iLuPb72OmX3YzCz+/WCi/buu4sF2kpntbmY9W34HPgU8U5+PWG4AAAOJSURBVLDar4Cz4qqFQ4EN7v56hUPttFLGmOXP0d3/H7DSzAbFi44FnitYLbOfYSnjy/LnV+AMip8WgjJ/hqoa6iIzmwTg7rcApwLnmtm7wBbgdM9Wx94/Ab+M/x/aFbjL3ecWjHEOcCLwIrAZ+HKVYu2sUsaY9c/xq8D0+NTCX4Av19hn2NH4sv75YWY9gE8CX2m1LNhnqM5iEZGc06khEZGcUyIQEck5JQIRkZxTIhARyTklAhGRnFMiEEkovrtlW3ef3Wl5Gf7eWDP7WKvn88yspufmlcpSIhBJv7HAxzpcS6STlAik5sTdw7+Jbzr2jJl9Pl4+0sx+H99s7sGWuzXG/8K+wcyeiNc/OF5+cLxscfw4qL2/WySG28zsT/H7T4mXTzCzWWY218xWmNl/tHrPOWb25zieW83sJjP7ONH9Zq616N70+8arf87MnorXP6JMu05ySp3FUouOB9a4+0kAZtbLzOqJbkB2iruvjZPDlcDZ8Xt2d/ePxzeguw0YDDwPHOnu75rZccC/A58tMYaLiW5DcraZ9QaeMrPfxa8NA4YD7wAvmNmNQDPwb0T3zdkIPAI87e5PmNmvgPvd/d54PAC7uvvBZnYicBlwXGd2lAgoEUhtWgZcZ2bXEH2BPm5mg4m+3H8bf5HWAa3vzTIDwN0fM7P3x1/ePYHbzWx/ojs71ieI4VNENym8KH7eDRgY//6wu28AMLPngL2BPsDv3f3v8fJfAP+rne3Pih8XAo0J4hLZiRKB1Bx3/7OZjSS6F8tVZvYQ0V1Gn3X3w9p6W5Hn3wMedffPmFkjMC9BGAZ8Np5tavtCs0OIjgRaNBP9f1jstsLtadlGy/tFOk3XCKTmmFk/YLO73wlcR3S65QWgr8Xz25pZve04YUnLdYTDie7kuAHoRTRNIMCEhGE8CHy11V0wh3ew/lPAUWb2ATPblR1PQW0kOjoRCUL/kpBaNITo4up7wFbgXHf/HzM7FZhqZr2I/tu/AXg2fs8/zOwJormoW64b/AfRqaELic7ZJ/G9ePtL42TwCtDmHLruvtrM/h2YT3Rf+eeADfHLdwO3mtn5RHfWFCkr3X1Ucs/M5gEXufuCKsexh7tvio8Ifgnc5u6Fk5aLlJ1ODYmkx3fNbAnRRDkvk9EpFiV7dEQgIpJzOiIQEck5JQIRkZxTIhARyTklAhGRnFMiEBHJOSUCEZGc+/+nCueHsUolSAAAAABJRU5ErkJggg==\n"
     },
     "metadata": {
      "needs_background": "light"
     }
    }
   ],
   "source": [
    "plt.scatter(df[:50]['sepal length'], df[:50]['sepal width'], label='0')\n",
    "plt.scatter(df[50:100]['sepal length'], df[50:100]['sepal width'], label='1')\n",
    "plt.plot(test_point[0], test_point[1], 'bo', label='test_point')\n",
    "plt.xlabel('sepal length')\n",
    "plt.ylabel('sepal width')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### scikit-learn实例"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "KNeighborsClassifier()"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf_sk = KNeighborsClassifier()\n",
    "clf_sk.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.0"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf_sk.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### sklearn.neighbors.KNeighborsClassifier\n",
    "\n",
    "- n_neighbors: 临近点个数\n",
    "- p: 距离度量\n",
    "- algorithm: 近邻算法，可选{'auto', 'ball_tree', 'kd_tree', 'brute'}\n",
    "- weights: 确定近邻的权重"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### kd树"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**kd**树是一种对k维空间中的实例点进行存储以便对其进行快速检索的树形数据结构。\n",
    "\n",
    "**kd**树是二叉树，表示对$k$维空间的一个划分（partition）。构造**kd**树相当于不断地用垂直于坐标轴的超平面将$k$维空间切分，构成一系列的k维超矩形区域。kd树的每个结点对应于一个$k$维超矩形区域。\n",
    "\n",
    "构造**kd**树的方法如下：\n",
    "\n",
    "构造根结点，使根结点对应于$k$维空间中包含所有实例点的超矩形区域；通过下面的递归方法，不断地对$k$维空间进行切分，生成子结点。在超矩形区域（结点）上选择一个坐标轴和在此坐标轴上的一个切分点，确定一个超平面，这个超平面通过选定的切分点并垂直于选定的坐标轴，将当前超矩形区域切分为左右两个子区域\n",
    "（子结点）；这时，实例被分到两个子区域。这个过程直到子区域内没有实例时终止（终止时的结点为叶结点）。在此过程中，将实例保存在相应的结点上。\n",
    "\n",
    "通常，依次选择坐标轴对空间切分，选择训练实例点在选定坐标轴上的中位数\n",
    "（median）为切分点，这样得到的**kd**树是平衡的。注意，平衡的**kd**树搜索时的效率未必是最优的。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 构造平衡kd树算法\n",
    "输入：$k$维空间数据集$T＝\\{x_1，x_2,…,x_N\\}$，\n",
    "\n",
    "其中$x_{i}=\\left(x_{i}^{(1)}, x_{i}^{(2)}, \\cdots, x_{i}^{(k)}\\right)^{\\mathrm{T}}$ ，$i＝1,2,…,N$；\n",
    "\n",
    "输出：**kd**树。\n",
    "\n",
    "（1）开始：构造根结点，根结点对应于包含$T$的$k$维空间的超矩形区域。\n",
    "\n",
    "选择$x^{(1)}$为坐标轴，以T中所有实例的$x^{(1)}$坐标的中位数为切分点，将根结点对应的超矩形区域切分为两个子区域。切分由通过切分点并与坐标轴$x^{(1)}$垂直的超平面实现。\n",
    "\n",
    "由根结点生成深度为1的左、右子结点：左子结点对应坐标$x^{(1)}$小于切分点的子区域， 右子结点对应于坐标$x^{(1)}$大于切分点的子区域。\n",
    "\n",
    "将落在切分超平面上的实例点保存在根结点。\n",
    "\n",
    "（2）重复：对深度为$j$的结点，选择$x^{(1)}$为切分的坐标轴，$l＝j(modk)+1$，以该结点的区域中所有实例的$x^{(1)}$坐标的中位数为切分点，将该结点对应的超矩形区域切分为两个子区域。切分由通过切分点并与坐标轴$x^{(1)}$垂直的超平面实现。\n",
    "\n",
    "由该结点生成深度为$j+1$的左、右子结点：左子结点对应坐标$x^{(1)}$小于切分点的子区域，右子结点对应坐标$x^{(1)}$大于切分点的子区域。\n",
    "\n",
    "将落在切分超平面上的实例点保存在该结点。\n",
    "\n",
    "（3）直到两个子区域没有实例存在时停止。从而形成**kd**树的区域划分。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# kd-tree每个结点中主要包含的数据结构如下\n",
    "class KdNode(object):\n",
    "    def __init__(self, dom_elt, split, left, right):\n",
    "        self.dom_elt = dom_elt  # k维向量节点(k维空间中的一个样本点)\n",
    "        self.split = split  # 整数（进行分割维度的序号）\n",
    "        self.left = left  # 该结点分割超平面左子空间构成的kd-tree\n",
    "        self.right = right  # 该结点分割超平面右子空间构成的kd-tree\n",
    "\n",
    "\n",
    "class KdTree(object):\n",
    "    def __init__(self, data):\n",
    "        k = len(data[0])  # 数据维度\n",
    "\n",
    "        def CreateNode(split, data_set):  # 按第split维划分数据集exset创建KdNode\n",
    "            if not data_set:  # 数据集为空\n",
    "                return None\n",
    "            # key参数的值为一个函数，此函数只有一个参数且返回一个值用来进行比较\n",
    "            # operator模块提供的itemgetter函数用于获取对象的哪些维的数据，参数为需要获取的数据在对象中的序号\n",
    "            #data_set.sort(key=itemgetter(split)) # 按要进行分割的那一维数据排序\n",
    "            data_set.sort(key=lambda x: x[split])\n",
    "            split_pos = len(data_set) // 2  # //为Python中的整数除法\n",
    "            median = data_set[split_pos]  # 中位数分割点\n",
    "            split_next = (split + 1) % k  # cycle coordinates\n",
    "\n",
    "            # 递归的创建kd树\n",
    "            return KdNode(\n",
    "                median,\n",
    "                split,\n",
    "                CreateNode(split_next, data_set[:split_pos]),  # 创建左子树\n",
    "                CreateNode(split_next, data_set[split_pos + 1:]))  # 创建右子树\n",
    "\n",
    "        self.root = CreateNode(0, data)  # 从第0维分量开始构建kd树,返回根节点\n",
    "\n",
    "\n",
    "# KDTree的前序遍历\n",
    "def preorder(root):\n",
    "    print(root.dom_elt)\n",
    "    if root.left:  # 节点不为空\n",
    "        preorder(root.left)\n",
    "    if root.right:\n",
    "        preorder(root.right)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 对构建好的kd树进行搜索，寻找与目标点最近的样本点：\n",
    "from math import sqrt\n",
    "from collections import namedtuple\n",
    "\n",
    "# 定义一个namedtuple,分别存放最近坐标点、最近距离和访问过的节点数\n",
    "result = namedtuple(\"Result_tuple\",\n",
    "                    \"nearest_point  nearest_dist  nodes_visited\")\n",
    "\n",
    "\n",
    "def find_nearest(tree, point):\n",
    "    k = len(point)  # 数据维度\n",
    "\n",
    "    def travel(kd_node, target, max_dist):\n",
    "        if kd_node is None:\n",
    "            return result([0] * k, float(\"inf\"),\n",
    "                          0)  # python中用float(\"inf\")和float(\"-inf\")表示正负无穷\n",
    "\n",
    "        nodes_visited = 1\n",
    "\n",
    "        s = kd_node.split  # 进行分割的维度\n",
    "        pivot = kd_node.dom_elt  # 进行分割的“轴”\n",
    "\n",
    "        if target[s] <= pivot[s]:  # 如果目标点第s维小于分割轴的对应值(目标离左子树更近)\n",
    "            nearer_node = kd_node.left  # 下一个访问节点为左子树根节点\n",
    "            further_node = kd_node.right  # 同时记录下右子树\n",
    "        else:  # 目标离右子树更近\n",
    "            nearer_node = kd_node.right  # 下一个访问节点为右子树根节点\n",
    "            further_node = kd_node.left\n",
    "\n",
    "        temp1 = travel(nearer_node, target, max_dist)  # 进行遍历找到包含目标点的区域\n",
    "\n",
    "        nearest = temp1.nearest_point  # 以此叶结点作为“当前最近点”\n",
    "        dist = temp1.nearest_dist  # 更新最近距离\n",
    "\n",
    "        nodes_visited += temp1.nodes_visited\n",
    "\n",
    "        if dist < max_dist:\n",
    "            max_dist = dist  # 最近点将在以目标点为球心，max_dist为半径的超球体内\n",
    "\n",
    "        temp_dist = abs(pivot[s] - target[s])  # 第s维上目标点与分割超平面的距离\n",
    "        if max_dist < temp_dist:  # 判断超球体是否与超平面相交\n",
    "            return result(nearest, dist, nodes_visited)  # 不相交则可以直接返回，不用继续判断\n",
    "\n",
    "        #----------------------------------------------------------------------\n",
    "        # 计算目标点与分割点的欧氏距离\n",
    "        temp_dist = sqrt(sum((p1 - p2)**2 for p1, p2 in zip(pivot, target)))\n",
    "\n",
    "        if temp_dist < dist:  # 如果“更近”\n",
    "            nearest = pivot  # 更新最近点\n",
    "            dist = temp_dist  # 更新最近距离\n",
    "            max_dist = dist  # 更新超球体半径\n",
    "\n",
    "        # 检查另一个子结点对应的区域是否有更近的点\n",
    "        temp2 = travel(further_node, target, max_dist)\n",
    "\n",
    "        nodes_visited += temp2.nodes_visited\n",
    "        if temp2.nearest_dist < dist:  # 如果另一个子结点内存在更近距离\n",
    "            nearest = temp2.nearest_point  # 更新最近点\n",
    "            dist = temp2.nearest_dist  # 更新最近距离\n",
    "\n",
    "        return result(nearest, dist, nodes_visited)\n",
    "\n",
    "    return travel(tree.root, point, float(\"inf\"))  # 从根节点开始递归"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 例3.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[7, 2]\n",
      "[5, 4]\n",
      "[2, 3]\n",
      "[4, 7]\n",
      "[9, 6]\n",
      "[8, 1]\n"
     ]
    }
   ],
   "source": [
    "data = [[2,3],[5,4],[9,6],[4,7],[8,1],[7,2]]\n",
    "kd = KdTree(data)\n",
    "preorder(kd.root)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "from time import clock\n",
    "from random import random\n",
    "\n",
    "# 产生一个k维随机向量，每维分量值在0~1之间\n",
    "def random_point(k):\n",
    "    return [random() for _ in range(k)]\n",
    " \n",
    "# 产生n个k维随机向量 \n",
    "def random_points(k, n):\n",
    "    return [random_point(k) for _ in range(n)]     "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Result_tuple(nearest_point=[2, 3], nearest_dist=1.8027756377319946, nodes_visited=4)\n"
     ]
    }
   ],
   "source": [
    "ret = find_nearest(kd, [3,4.5])\n",
    "print (ret)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:2: DeprecationWarning: time.clock has been deprecated in Python 3.3 and will be removed from Python 3.8: use time.perf_counter or time.process_time instead\n",
      "  \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "time:  5.170202400000001 s\n",
      "Result_tuple(nearest_point=[0.09916902877403755, 0.5005978535517558, 0.7997848590100571], nearest_dist=0.0010460533893058112, nodes_visited=38)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\ipykernel_launcher.py:5: DeprecationWarning: time.clock has been deprecated in Python 3.3 and will be removed from Python 3.8: use time.perf_counter or time.process_time instead\n",
      "  \"\"\"\n"
     ]
    }
   ],
   "source": [
    "N = 400000\n",
    "t0 = clock()\n",
    "kd2 = KdTree(random_points(3, N))            # 构建包含四十万个3维空间样本点的kd树\n",
    "ret2 = find_nearest(kd2, [0.1,0.5,0.8])      # 四十万个样本点中寻找离目标最近的点\n",
    "t1 = clock()\n",
    "print (\"time: \",t1-t0, \"s\")\n",
    "print (ret2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 第3章 k近邻法-习题\n",
    "\n",
    "### 习题3.1\n",
    "&emsp;&emsp;参照图3.1，在二维空间中给出实例点，画出$k$为1和2时的$k$近邻法构成的空间划分，并对其进行比较，体会$k$值选择与模型复杂度及预测准确率的关系。\n",
    "\n",
    "**解答：**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.colors import ListedColormap\n",
    "\n",
    "data = np.array([[5, 12, 1], [6, 21, 0], [14, 5, 0], [16, 10, 0], [13, 19, 0],\n",
    "                 [13, 32, 1], [17, 27, 1], [18, 24, 1], [20, 20,\n",
    "                                                         0], [23, 14, 1],\n",
    "                 [23, 25, 1], [23, 31, 1], [26, 8, 0], [30, 17, 1],\n",
    "                 [30, 26, 1], [34, 8, 0], [34, 19, 1], [37, 28, 1]])\n",
    "X_train = data[:, 0:2]\n",
    "y_train = data[:, 2]\n",
    "\n",
    "models = (KNeighborsClassifier(n_neighbors=1, n_jobs=-1),\n",
    "          KNeighborsClassifier(n_neighbors=2, n_jobs=-1))\n",
    "models = (clf.fit(X_train, y_train) for clf in models)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA2cAAAE/CAYAAADCCbvWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nOzde3SbV3rf++8GQRIkQRLUlZIoCbJ1oWTZA9myTc/IM8jMJOPcmqRx0s6k6aBN4/S0OeewzVmrbv4Jpj1t1HNyQdZJmmTSZGHSZJJmOUnTpMmkk8trR7bpsWzDtmxTliy9omgJkigJpCAKJEHs8wdAiDdJJEXyBcDfZy2sAV7cHtAabDzv3vt5jLUWERERERER8ZbP6wBEREREREREyZmIiIiIiEhFUHImIiIiIiJSAZSciYiIiIiIVAAlZyIiIiIiIhVAyZmIiIiIiEgFUHImFccYkzXGPLDAx1pjzO473Bczxhxb3uiWjzHmR4wx/+su90eNMYOLeD3HGPPPlic6ERGRmTQ+l+/X+CwrRsmZ3JMxxjXGfH7a7X9ojLlujPnMPI+Nlr6Qf2XW8WPGmNhC3s9aG7TWnrnvwCuctfZ3rbXfMXX7bgPZajHG/LAx5hVjzKgxxvEyFhERuTuNzyujQsfnnzPGnDLG3DDG9Btj/rGX8cjKUXImi2KM+TLwK8B3W2tfvMPDbgL/2BgTXq24Vpoxxu91DKvkGpAAjnodiIiILJzG55p3E/heoB34MvBLxphPehuSrAQlZ7JgxpjngJ8HvmCtfeUuD80ASeBn7vJa/9QY80HpDN9fGmN2TruvfIbKGLPeGPOnxpgRY8zrxpj/e56lEJ8vnU26boz5FWOMmflW5v8zxgyXzjR9btodW40x/8MYc80Yc9oY8+PT7osbY14wxvyOMWYEiBljnjDGHC/FcskY8wt3+GwvGmN+sHT9SOnzfFfp9ueNManS9fKyDmPMS6Wnv11aNvIPpr3eTxljLhtjLhpj/skd/+ozY9hijHnHGPN/LeTxU6y1f2Wt/QPgwmKeJyIi3tH4vCbG55+x1vZbawvW2teAvwOeWsxrSHVQciYL9b8B/x74nLX2+AIe/x+AHzTG7Jt9hzHm+4GfBv4+sJHiF8zv3eF1foXi2aJOimeKvjzPY74HeBz4BPDDwBem3fckcAbYQHEw+iNjzLrSfb8HDAJbgWeB/zh9cAC+D3gBCAG/C/wS8EvW2jbgQeAP7hDzi0C0dP3Tpff/zLTbc85oWms/Xbr6idKykf9Wut1J8SzZNuDHgF8xxnTc4X0BKJ0RfRH4ZWvtz5WO/WdjTOYOl3fu9noiIlLRND6vsfHZGNNE8e/63t3eT6qTkjNZqG8H+oB3F/Jga20a+DXg381z908AP2ut/cBamwf+IxCZfnYOwBhTB/wg8DPW2lFr7fvA1+Z5vaPW2oy1dgD4WyAy7b7LQMJaO1H6Qj0JfLcxZjtwBPg31tqctTYF/BfgR6c991Vr7X8vnaW6BUwAu40xG6y1WWtt3x0+/ovM/LL/2Wm3P8M8X/53MQH8u1L8fw5kgTkD6jQHAIfi3+yrUwettf/CWhu6w+WRRcQjIiKVRePz2huffw14G/jLRcQrVULJmSzUPwf2Av9l1rKEu/lPwBeMMZ+YdXwnxbXSGWNMhuI+J0Px7NN0GwE/cH7asfPMlZ52fRQITrv9sbXWTrt9juKZuK3ANWvtjVn3TY9h9nv9GMW/QX9pCcf3zBMLwKvAXmPMZooD0W8D240xG4AngJfu8Lz5XC0NkFNmf77ZfgT4mOIZRRERqX0an9fQ+GyM+X+Bg8APz/r7SY1QciYLdRn4HPA08J8X8gRr7VWKxSX+/ay7zgM/MevsUNM86+SvAHmga9qx7YuMe9uswWoHxf1UF4B1xpjWWfd9PP0jzPo8p6y1XwQ2URzYXjDGtMx+Q2vtKPAG8H8CJ6y148ArwL8GPrLWDi3yMyxGHBgCvl46swmAMebXSmvl57toWYSISPXS+LxGxmdjzFeA7wS+w1o7soKxioeUnMmCWWsvAJ8FnjHG/OICn/YLwCeB/dOO/Rrwb40xDwEYY9qNMT80z/tNAn8ExI0xzcaYbmCxpWM3Af+HMaa+9B77gT+31p6n+IX8s8aYgDHmEYpn3n73Ti9kjPlHxpiN1toCxU3VAJN3ePiLwE9ye4mEM+v2fC4BC+ofcxcTwA8BLcB/Ncb4AKy1/7y0Vn6+y0NTTzbG1BljAhTPiPpKf5v6+4xJRERWkMbnNTE+/1vgS8C3l5JrqVFKzmRRSl+anwWeNcb87AIePwL8P8C6acf+mOKZrd83xUpLJyieCZrPT1LccJsG/ivFTcJjiwj5NWAPxbNV/wF4dtqX2heBMMWzdH9McR34N+/yWs8A7xljshQ3H/9Da23uDo99EWjl9hKJ2bfnEwe+VlpO8sP3+Fx3VDoT+PcpDny/NTUALNCPAreAX6V4FvYW8BtLjUVERFaHxueaH5//I8UZxFPTZtZ+eqmxSOUyWq4q1cQY85+ATmvtfFWhRERExAMan0WWh2bOpKIZY7qNMY+YoicoLm34Y6/jEhERWcs0PousjLXSVV2qVyvFpRJbKW56/nngTzyNSERERDQ+i6wALWsUERERERGpAFrWKCIiIiIiUgGUnImIiIiIiFSAVd1z1tzebEOdodV8SxGRNevihxeHrLUbvY5DKt+G5mYbDq3S+Hz1Kql141BXx6aWTavzniIiFeRu4/OqJmehzhDP/fpzq/mWIiJr1le+7SvnvI5BqkM4FOL4c6s4PieThJ8dJLNhlN6e3tV7XxGRCnC38VnLGkVERGR1xWK4L3RB7k59gkVE1iYlZzXIWsvwpWEG3h3g4w8+ZvzWuNchiYiIzMtxHa9DWFW3btzi/HvnGXh3gOy1rNfhiEiFUZ+zGpMfz/PGN9/g4tBF7HqLyRt8r/iI9ETY8dAOr8MTEREpCocJZQdJuX0ARMNRb+NZYdZaTr1+ig/e/QC70YIBjsOu8C4e/vTD+Op0vlxElJzVnPdffp8LExcIfTqEMQaA/GieN7/1Jq0drXRs7fA4QhERESAaxXUg2t1HitpP0C5+eJETH56g/el26hrqALCTlo+Of0RLqoXdj+32OEIRqQQ6TVNDxm+N455xaX+ovZyYAfib/dSF6zhz4oyH0YmIiMwSjeJ8o5MQAa8jWXGn3j1F096mcmIGYOoMrQ+1curEKWzBehidiFQKJWc1JJfNQQB89XP/szaua2T4+rAHUYmIiNxDLoebcb2OYkWNZEZoXNc453h9sJ7x/DgTYxMeRCUilUbJWQ1pbGnE5iyFfGHOfePD4wTbgh5EJSIichexGOEMZIYGSaaSXkezYlpaWxgfnlugKz+ax+/z42/QThMRUXJWUxqbG+na3sXIhyNYe3t5xOT4JBNnJ3jgoQc8jE5ERGR+Tufz9J4Iksmka7Z6456Dexj9cBQ7eXt8tgXLSP8ID3Y/qIIgIgKoIEjNefjph7n5P29yte8qvg0+7ISFS3Dw4EHWb1/vdXgiIiLziqdCJCMZr8NYMV37u7h2+Rpnj52FzWB8BnvJsm3dNvYc3uN1eCJSIZSc1ZiGpgaO/MARrp6/ytWLV6lvqKfzqU5aOlq8Dk1EROTusllS6VRNVm00PsMj0UcIXwlz2b2MtZYND22gY2vHjCJeIrK2aQ69BvnqfGwMb6T7qW4efOxBJWYVqDBZoDA5d2+giMiaFYsROx2EkRESfQmvo1kRxhjaN7Wz54k97H1yL+u2rVNiVmFswWp8Fk9p5kxkFY1cGeHk8ZNcOH8BgK07ttJ9uJvWDa0eRyYi4r14pJd4MkkoVtx7VoszaFKZxm+N8+HrH+J+6JKfzNOxvoPuR7vZ/MBmr0OTNUYzZyKrZOTKCC/92Uukm9K0fraV1s+2kg6keelPX+LG1RtehyciUjHWQt8zqRz58Twv/4+XOX3jNE1PN9H+He2M7hjllRdfYbB/0OvwZI1RciaySk6+cRK7y9IabsXn9+Hz+2gNt1IIF/jwjQ+9Dk9EpHJks6Tcvpqt3CiV5eOTH5OpzxB6KERdYx3GGJo2NdHyWAvvvvauljnKqlJyJrIKCpMFLgxcoKVr7v6/lu0tDJ4dnNH+QERkzYrFcF/oIpTNK0GTVTF4dpBA19zZ2oa2Bsbrxxm5MuJBVLJWKTkTWQXa8C0isghTCZqWN4rXdN5UVpmSM5FVYHyGbTu2cfP8zTn3ZQeydO3qUgInIjJbLud1BLIGbN+1ndz5uf/WxofHaZxspG1jmwdRyVql5Exklex7fB8+18fImREK+QKFfIGRMyPUnatj3+F9XocnIlJZwmHCGUidPkYylfQ6GqlhW/dtpWOyg8y7GfK38tiCZTQ9ys03b/LIk4/gq9PPZVk9+tcmskpa17fyme/7DNvy28j+TZbs32TpmuziM9/3GYLrgl6HJyJSWaJRnP4eIkN+MkODStBkxfgb/Hzyez/JvnX7GHt1jOFvDtN2sY2nP/c0W/dt9To8WWPU50xkFQXXBXnsOx7jUfsooL1oIiJ3FY3iECWeSpA4WEzQYpGY11FJDWpoamD/p/bT/clusMXtCCJe0MyZiAeMMUrMREQWKB7ppfdEUDNosuKMMUrMxFP3TM6MMQFjzLeMMW8bY94zxnyldHyXMeY1Y8wpY8x/M8Y0rHy4IiIiAmtvfI5nIkSyWgIuIrVtITNnY8BnrbWfACLAM8aYHuA/Ab9ord0DXAd+bOXCFBERkVnW3vicy5HJZbyOQkRkxdwzObNF2dLN+tLFAp8FXigd/xrw/SsSoYiIiMyx5sbnaBTnG50wMkKiL+F1NCIiK2JBe86MMXXGmBRwGfgm8BGQsdbmSw8ZBLbd4bnPGWOOG2OOX75ymWQqqfXiIiIiy2C5xucro6OrE/D9isXIfH0HoUxOvyVEpCYtKDmz1k5aayNAF/AEsH++h93huV+11h621h5uqoewm9GGXhERkWWwXOPzxubmlQxzeYXDhHMBr6MQEVkRi6rWaK3NAA7QA4SMMVOl+LuAC/d6/r7RJpxUZEbFJc2kiYiI3J/7HZ+rTi5HJpPGcR2vIxERWVYLqda40RgTKl1vAj4PfAD8LfBs6WFfBv7knu/W2grRaLkkbrg/Tbg/TebygNaPi4iILMKyjs/VpNScOpTNk3L7lKCJSE1ZyMzZFuBvjTHvAK8D37TW/hnwb4B/bYw5DawHfnMxbxyP9OJ0Po/T+Ty977dpg6+IiMjirMj4XBWiUdzjR4hkArgZ1+toRESWjf9eD7DWvgMcmuf4GYrr2+9bPNJLPJkk9KXiDFpvT+9yvKyIiEjNWo3xWUREVtc9k7NVE4uRSSYJPztIwjkKgeJm30hnhGg46m1sIiIiUnlyufIe9lgk5nU0IiL3bVEFQVZcLIb7Qhe9qQC9fRAaymo9uYiIiMwVjeJ0Pk9kyK8q0CJSMypn5mxKLEa8dDXuOIQPHyPl9gFoBk1ERERmcPp7iEZSuBu8jkRE5P5V1szZbFMbftOQOn1MM2giIiIyVy7ndQQiIsuispMzKJfMjQz5SZ0+RsI5SsI5quULIiIiAtEo0XRAbXlEpCZUfnIG5XXlmd/pIpPspDdV/BJWgiYiIiLxSG+5LY9+G4hINauO5GxKLFbck1b6EtZZMhEREYFigha51uB1GCIi96W6krNppp8lS/QlcFxHe9JERETWuEwu43UIIiJLVrXJGRQTtMzXdxAayuL29xX3pGkmTUREZE1yvtFJaChb7JcqIlKFKq+U/mLFYriOU7zuuuUm1qFQJ+FQGFAJfhGpHSNXRjjff54bIzdo72hnx/4dtHS0eB2WSGWIxXCTyfJvgUi4R78BRGRV5LI5BvsHuXr5KoGmANv3bqdjawfGmEW9TvUnZwDRaPmqm0wSfSYNmQxuoI9MoPQQfTmLSJUbeG+AN7/1Jr4uH/71fi5lLnHqj0/xZPRJNj+w2evwRCpD6aSt+qSKyGoZvjTMsT8/Rn5Tnvr19eRH85z9q7N07+1mX8++RSVotZGcTReL4UzNpAHR7j5S+WPF6/pyFpEqdWvkFm+99hbBniD+5tJXdyeMbxnnded1nul6Bn9D7X2liyxJNIrrlH4DBFIa/0VkxdiC5fhfH8fsN7R3tpePF7YX6H+ln807N9OxtWPBr1fVe87uKBotX6b3SFN5XRGpVhc/uojdZG8nZiUN7Q3k2/Ncca94FJlIhSr1PxMRWUnDl4e5YW/Q3Nk847iv3odvu4+BkwOLer3aTM6mm5agZYYGSaaS5YuISLUYuzWGr+kOX9kBmBibWN2ARKpFNqsxX0RWzERuAhOYf9miv9lP7lZuUa9X+8kZlJtY954IEu5PE+5Pq4m1iFSVjk0dFK4V5hy31sI1aF3f6kFUIpUtHumdcXJWRGS5BdcHsRlLIT93jB6/Ms76zesX9XprIzkriUd6cTqfLyZqamItIlVkY3gjrROt3Dhzo5iQUVznPnJyhA0tGwhtCXkcoUhlmjo5q/5nIrISmlqbCIfDDJ8YLido1lpG06M0XG2ga1/Xol5vTSVn081uYi0iUsnq/HU89d1PEcqEGH5pmOE3hhl+cZjN+c08/szjiy7VKyIiIsvj4U8/zK7WXYy8OMLIGyOMvDxC49lGPvVdnyIQXNze1zVd2ise6SWeTBL60swZtN6eXg+jEhGZX3N7M0d+4AjZq1ly2RzN7c3qcSayUKWTsRrjRWS51dXXEflchH039pG9lqW+sZ72ze1LOnG6ZmfOymIxMl/fQW8f9PZBaChLwjmK4zpeR1bzCpMFMukM1z6+Rn4873U4IlXBGEPrhlY2hjcqMRNZIK2WWRxrLTeGbnB18CpjN8e8DkekajS1NrFx50ZCnaElr2hZ0zNnZbEY8dLVeDJJ+NlBNa5cYemP0qReTjFWNwZ14L/l58ChA4Q/EdbyLBERWXbxSC+kEiR6vI6ksmWvZXnjb97g+s3rmCYDNyD8QJiDRw5SV1/ndXgiNU/J2WyxGK7jED58TAnaCrl+4Tp9L/XRdKiJ9lCxWV9+NM9bb7yFv8HP9gPbPY5QRERk7Rm/Nc7Lf/YyE7smaO8qLskq5AucefcMhRcLHPr8Ia9DFKl5WtY4n2gU9/gRImlInT5GwjlKwjmqMrzL5FTqFP4H/TSGGsvH/M1+Wh5u4YM3P8AWrIfRiYhIrYqnQtq+cBcXPrzArbZbBLcHy6tYfH4foUdCnDt3jtHhUY8jFKl9Ss7upNS8OvM7XWSSnUTSlPukOK6jL/X7cCV9habNTXOON4YaGR0bZTw37kFUIiJS82Ix3Be6CGXzpNIpr6OpOEOXhqjfWD/nuKkzmA7Djas3PIhKZG3Rssa7iUbLVx0gnkqQ3D2Im8mQIUcqnVLVpyVoDDQymZukrnHm2vXCRAFfwUedX2vaRURkhcRixFIJEhu8DqTyTI3P8xoDf4N+NoqsNM2cLUI80ov7QhduMkRvKqCqT0u0a98ubp6+WW6kO+XG2Rt0hbv05S8iIisvl9MqmFm69nRRGCxQmCjMOJ4byhGYCNCxpcOjyETWDiVnixWLFas7qizvku18eCeb6zaT+VaG0Yuj3Lp0i0wqQ3AoyIGnDngdnoiI1Lh4JlLeV6795LeFOkPs27OP4VeHuXHuBrmhHMP9w0y8O8Hjn3scX51+NoqsNE1R3IfZTaxDgRAAsUjM28AqnL/Bz5Pf8yTpj9KcP32eQqHA1ge2sm3vNuoDc9e6i4iILKtoFMeBaHcfKX/a62gqhjGG7qe62bR9E+dOnuPWx7cIbwqz46kdNLc3ex2eyJqg5Ox+xWJkkkmiz6QhkMEN5Eg4R4mEe1SC/y7q/HVs27eNbfu2eR2KiIisRdEoTtIlFFNyNp0xhvXb17N++3qvQxFZk5ScLYdYDMdxyjfVI01kaQqTBT7u/xj3Q5fxsXE2b93Mrod30dLR4nVoIlKr8nmSqaRWvYjchbWWoYEhzpw4Q/ZGlrZQGw8cfID1XUril5sWDy+XaLR8md4jTZuNRRamMFng9b94nddPvE52W5bJA5N8lPuIv/3vf8v1i9e9Dk9EalEsRu+JIJnLA9o/LnIXp4+f5tiLxxhqH2LywCSXmi/x0l+9xNm3z3odWs1RcrYSSj3SIkP+8mbjqYuIzO/iqYt8fPNjOp7ooGlTEw1tDbTvbce330fqpdSc6p4iIstBBb5E7u5m5ibvv/s+bT1ttHS10NDWQHBHkNYnW3n3+LuM3RzzOsSaouRspUxL0ML9acL96XITaxGZ69ypcwR2BjDGzDjetLmJ4dwwN6/f9CgyEal1StBE7uzS2UvYzZa6hpl9aP1NfgobClw5d8WjyGqTkrOVFI3idD5fvkwtnVCCJjLXxPjEnC9+KG5ONw2GyYk7NEYVEVkG8Ugvma/vUP8zkVnyE3lMvZn3PltvNT4vMyVnq2jqzJzWtovM1dnVSe5ibs7x/Ggef86voiAisipCBLwOQaSirOtch71i52wvsNZihgztm9s9iqw2KTlbZVo6ITK/nQ/tpOFKA9mBLLZQHAAmbk4w8tYI3Z/oxt+g4rIisgqyWVJuH4m+BIm+hGbRZM1b37WeDc0bGH5vmEK+AEBhokDmnQxb1m9RcrbMlJx5oLx0QgmaSFkgGODI9x4hdDXE8IvDDB8bZuL1CSIPRXjg0ANehycia0EshvtCF72pAL19EBoqJmpK0GQtMz7Dk9/5JDubdpJ9Mcvwy8NkX8ryYPuDPPbtj83ZKy73R6eivRKL0ZtKkOjxOhCRytG6vpUjP3CEWyO3yE/kaW5vps4/dx+aiMiKicWIl67GHYdodx+p/O3+pVNCoU71RpM1oz5Qz6HPHeKhWw8xdnOMQDBAfaDe67BqkpIzr+Xm7rGR2jZ2c4xrH18DYN22dTS2NHocUeVpamvyOgQRkWJhLwdw3ZmHn0mTyg+qeXWNmZyY5OrgVfLjedo2thFcF/Q6pIrT0NRAQ1OD12HUtHsmZ8aY7cBvA51AAfiqtfaXjDFx4MeBqfqZP22t/fOVCrQWxVMhkrsHSThHiYR7iIajXockK8hay6nXT9H/bj92ncVi8f2dj/2P7Gf34d1aFiAii6LxeZVEo3MOOUA8lSBxYEAJWo24dOYSx188zkRwAhqAV6BrSxeRz0a051lW1UL2nOWBn7LW7gd6gH9pjDlQuu8XrbWR0kVf/IsVi+EeP0Iom5+zXEJqz8cffMyJD0/Q8nQL7YfaCR0K0XKkhXdPvsuFkxe8Dk9Eqo/GZw+pAnPtyF7L8tqLr1H/aD2hx0OEPhGi/TPtDOYGOXHshNfhyRpzz+TMWnvRWvtm6foN4ANg20oHtmZEo7gvdHkdhawway0n3z5Jy4GWGb286hrraN7fzMm3T3oYnYhUI43P3lMF5tpw7r1z2C5LQ/vt5XrGZ2g72Ma5s+cYuznmYXSy1iyqWqMxJgwcAl4rHfpJY8w7xpjfMsZ0LHNsa0s+j+M65YvUFluw3Bi5QUPH3HXajesaGbk+Ui4fLyKyWBqfvaMKzNXv2tA1GtfN3f/t8/sgCKPDox5EJWvVgpMzY0wQ+EOg11o7Avwq8CAQAS4CP3+H5z1njDlujDl+ZVT/uOcVDhMZ8uP29+H295E6fYxkKul1VLKMjM/Q0NhAfjQ/5778zTyBpgDGpz1nIrJ4Gp8rQCxG5us7CA1lSThHvY5GFqkl2MJEdmLOcVuw2FFLQ7MKYMjqWVByZoypp/jF/7vW2j8CsNZestZOWmsLwG8AT8z3XGvtV621h621hzc2Ny9X3LUlGsXpfB63rwe3r4feE0Eyl4ubjKcuUt2MMew+sJsbJ29g7e0ZMluwZE9mefDAgx5GJyLVSuNzBSn1SAtl8ySco1oFU0V2du8k7+YpTBRmHM+6WTZ2bKQl1OJRZLIWLaRaowF+E/jAWvsL045vsdZeLN38AUA7Ju9XqSJUnCikEjjX0hAIkApmSfQl6O3p9TQ8uT8PHnqQa5eucfHVi9R1FvedFS4W2BraqibLIrJoGp8rUCyG6ziED9/ui6ZKzJVvXdc6Htr/EO8fex+z1eAL+JgcmiSYC3Loew55HZ6sMQupDfop4EeBd40xqdKxnwa+aIyJABZwgZ9YkQjXqHikl2JzFYiHUiQOFNeyhwIhAJXtrUJ19XU8+d1PcnXwKhfdi/iMj81Pb2Z913otaRSRpdD4XImiUVyHcvNqUIJW6Ywx7H1iL527Orlw+gLjY+OsO7COzgc771pGX20UZCWY6UusVtrhrVvt8eeeW7X3qynJJNFn0gC4wTyZoJ/e6PMeByUilewr3/aVN6y1h72OQyqfxucV4DjFBG1DntCGLv2IrzGJvgRks+D3q1etLNrdxmd11asWsRjO1PVkkvCzxebVoVAnAOFQuCa/GAqTBQZODHD6vdOMZkdpX9fO3k/spXN3p5o2i4hI5YpGcZzSDBqDNTnLcvnsZT5Mfci1K9cINAfYfWA3Ox/ZSZ2/7t5PrjKO6+BmXAAymTShbB73hS7Czw5qCassq0WV0pcKUWpeHUlDuD8Ng8UvhlrbfGyt5a2/eos3+99k8sAkwc8GGd0xyquvvsqZN894HZ6IiMjdlQp+RYb8ZIYGa6rA17l3z/HySy8zsmWE4GeD8DCkzqQ4/pfHKUwW7v0CVcRxnWICNjhIuD9NJE2xR23p91gom6/J32HiDc2cVatoFIdo8frU0on8MVLp4raDUCBU9Wforl+4zvnL5+n4VEd5T1bTpiYa2hs48Xcn6NrfRWPz3L4kIiIilcTpfJ54KkHiwEBNzKBNjE3wzrfeofWpVvzNxZ+SDe0NdDzawYW+C1w9f5WN4Y0eR3l/kqkkmVymeCObJTLkx+k/Ui7eRqz0wKk9hpFUeWZN5H5o5qwWRKM4/cUS/L19EHFzNXGGLu2mMVvMnGIZdY11sB6unr/qUWQiIiKLE4/00vt+G5nLA1XfrPr6hesU2grlxGyK8Rn8W/1cOHvBo8iWRzKVJDM0SMTN0dsHvSeCOP09txMzkRWkmbNaEY0WS/CX1MIZOmvtHasYWrN6hWxERESWQzzSC6lEuQLzlGprlWOtvePpfR+L/mcAACAASURBVOMzc/qFVYN5Z8o6n4fOBb5A7vaJ8Wr93SWVQTNnNaoWztBt2r6JyfQksyuKFiYKmKuGjq0dHkUmIiKyNPFIL5mv7yjOyPRBaChLwjnqdViL0rGlA5MxTI5NzjhurSV/Mc+WnVs8imxpEn0JMpcHyv9Nek8Ei4nZQtXw3kJZfZo5q2Gzz9BV25m5Dds30BnsJP12mtZ9rfib/IyPjJN9L8u+7n00tTZ5HaKIiMjixWLES1fj0yowT6n00uwNTQ10P9zNieMnaHmohcZQI5O5SUZOjbDBv4FNuzbNeHyiLwG53D1fdzU/94yYxsfpfb+t+LvpPtTa3kLxhpKzGlfNCZrxGR5/5nFOHT/FmVfPkC1kaWps4tFHHmXnwzu9Dk9EROT+xWK4jgOuC1A1pdn3PL6HQEuA/rf6yeQy+I2fvfv2svfxvfjqiguzpqocTpWdv5vV/NyJvgSMjJD5+o7bB2OxZXnt8u8ulKDJ0ig5WwPikd7bZ+aqLEHzN/jZ/8n97HtyH5P5Sfz1/jvuQxMREalK0wpNuNMrMJeSlVCos+J+5Btj2PHQDrYf2E5+PE+dvw5fne922fmSUDaPe/wIxKJ3fT3XcQgfPrYiCdq8Mb2wY9kSstnikV6c9FHcTfd+rMhsSs7WilgMd9rSiUpfMjGbr85XPhMnIiJSs0rNq6dm0qLPpEnli/uYwqFw8VgFjd/GGF6++HL5dur0sWIxjW+UKmmEwwurcjhVkr6UmM64awmfd3rPsXln71YoMZuuXGBEZBGUnK0l0xK0algyISIisiZNS2YcihWYk7sHcTMZMuRwM27FzKQlU0kymTQhApDLEcn4i2Xn7zFTNq9SYhrt7sPN90EgsKTPOzcmij3KlhLTEjnf6KzKFUviPTO7Et5KOrx1qz3+3HOr9n5yB6WlA5mgv+pm0ERk4b7ybV95w1p72Os4pPJpfK4CySQA8UiGxIERaGsj0hkBVu9E6/TZKAA34xarHL7fRjwVKh5crhmpWZ83tGnHXWcOp2JzM26xR9lSZu+WW+mEeGZDUAmazHC38VkzZ2vRPEsHlKCJiIhUsFLSEwdIJUjuzuJm+sj486TSqRX/8Z/oS0A2Syg/7adjPk/v6VKVw8gyv+Hszzs+iJtOz/t5p5pGT8UWyZR6lMWWOabFisWIpRIkNngch1QVJWdr1bSlAymOVdQSiUriuI4SVxERqSjxSC9xxynecN3yfvJQqDhTFA6Fl2XsmurXlcllblc3DIdnPmgVZqSmf954KFWuQB0KhMqx9b7fRjwTWbWYFiWX0+8JWTAlZ2tZNIpDlGj6KCnU1X66cmWn/OqckRQREVmU6RUek0miz6Qhk8EN5Ehl0sWHLDEZKJfAz0E4V9y35Xxj5aobLkjp88aJEp/6vIEM5HJE0/ffo2ylxDMRnHQfKbTXXxZGyZmoaWKJ4zq4GReguJE4myd2OjjjDB2wZv8+IiJSoWIxnKmZNCiXpJ8a0xYykzbfGOi+0HV7pmwVi2nc06zPW3EzZdNNrVSiuJVEK5XkXpScCTCzaeJarCw0tV49MnT7/xJO/5HiF34qgdOZA9Kk1o2vyb+PiIhUuBm90orbFkgXZ9BSGwbvmhQ4rlMugV8+tsrVDRetkhOy2WZsJdFKJbk7JWdSVk7QSjNFtZ6ATK2lB25Xdurvuf2FXyryNGOpxLReccu9tl9ERGRZlLYtlG9O274wn7uNgbJMpm8lCar/mdyZuvrKDPFIb3HD70gxQatVyVSSzOUBwv1pwv1pek8Ei5Wd7nUmLhbDfaGLSBrC/WkYLPaMm11eWEREpFI4nc/TeyJYHvNmXxY8Bsp9i6YDXocgFU4zZzJXLEZm2gxRb/R5ryNaFjOSzanKTlOzYos5QxiL4Uxddxy1JBARkYp314IZmiVbXSNrY4WSLI1mzmR+pRmiUDZPwjla9TNDib5EMSHro3h5f5kqO0WjOP09RIb8pE4fu+OSEREREZF4pJfe99tqfoWSLJ1mzuTOYjHc0gxayq2+ErCzZ8oyX1+hMsCzNvpOvW8oENKGXxEREZmhvMe/x+tIpBIpOZO7i8Vwp5buVUmPjnJ/llIpfIB4aoX7s5Q2+sZTtxPCtd6aQEREREQWR8mZ3FsV9OgoN40uCWXzuMeP3N7cHFmdOGYslZxqTeAcLd4OBLS+XERERIinQiR319beflke2nMmCzNtb1Vm6M7leL2QTCWL/VnSkEl2kkl2zkzMPDJV+TKT7KQ3FdD6chERESmqsb39snw0cyYLN71Hh8dNFJOpJJlMmhAByOWIZEr9WSqtYWZpKWUciE9VwFSFJhEREanyvf2yMjRzJos21S8lc3lgVWfQHNfBcZ1yj7LeVAA3GcL9/c7q6M8ydZZsKFs+SzZ1ERERkTUoFsM9foRImhnbM2Tt0syZLEm50hADqzITlOhLQC5XnCnLZuk9XSqFv0p7yZbNtLNkrpsCIEMO0NkyERGRNSkaxUm6hGJpryORClCVydnI2BhvfvwxV4aHaWtu5lBXF5taWrwOa80pJ2gHVqaZ4tSMUiqdut00OhUCQitbeXGllSpg4roARJ9Jl5tYT1GiJiLVaHxyknfSadzLl2loaOChLVt4oKMDY4zXoYmIVIWqS84+unaNP3z1VQ5OTNDt93NlcpKv9ffzbY89xuGuLq/DW3PikV7iySShLy3vDFrCOUqoOKFEKJ/HfaFUCr/aZsruZNoSTKfUqsDNF5czZPz5iqyIKSJyN8O5HMlXXmHzyAj7/X5uFQp849QptjzwAN//8MP4lKCJiNxTVSVnE5OT/NG3vsU/qKtj57SZskMTE/zGG2/wwPr1rGtq8jDCNSoWIzNV7GKJJWEd18HNuADFQh/ZPO4LXRAOl94jumzhVpxSoRUcp3iz1MxaPdJEpJr82bvv8uiNGzwdCpWPPVYo8NunT/P2xo0c2rLFw+hEKpjjEH1GSxqlqKoKgpy8epUtuRw7A4EZxzvq64lYS+rCBY8ik/spCTvVoyzsZgi7GSJpiqXwY7HiDFOlF/pYLqXP6lXBFRGRpboxNsbghQv0tLbOOF7v8/F0IMBbZ896FJlIhSutnkltyBMKdXodjVSAqpo5y46Ps/4O962vq+PjW7dWNR6ZpbSXKnz42D1Lwk5POjJDg0SG/Dj909YsrpWE7A5mF1wJBYpnojWTJiKV6ObEBG0Uk7HZ1tfXkx0dXf2gRKqB65KKQiR8RPvNBaiy5GxzSwtvAdbaOZuL3clJutrbvQlMbotGcZ3S0rxSkYvZXzbJVLKckBWVepSt8YRstqkEzenMAWlS68bVI01EKlJHIMBwXR3ZyUmCdXUz7nNzOTZv2+ZRZCJVIBBQYiZlVZWchUMhzPr1HLt2jSNtbeUE7YObNznb1MR3az17ZYhGcZypvVPHynvJpmQuDxQrL0amJRmayZ/XjL/RtH19U0sfwqGwvtBFxHONfj+R3bv50w8+4Nn29vIM2tDEBC8WCvzgAw94HKGISHWoquTMGMMXH3+cP3jjDVJXrrDdGK5Yy83WVr70xBME/FX1cWpbKUGLh1I4nbc3uUbTAWBWYiYLU+qRFo9kcDrTuME8qUzxb6sETUS89vm9e/nT8XF+8cwZdhvDLWDQ7+cLTzzBzmlFQkRE5M6qLptpDwT4Z5/8JIMjIwyNjnKwsZEHOjpUorcSRaPEic48phmy+xOLEZ+6vsD9fSIiq8Hv8/EDjzzCtT17GBgept7n49l162jUiVOR+ZVWxEDQ60ikglTlN6Yxhu3t7WzXHjNZooK1XMpmsRT3MtbNs4m94i1gf5+IyGpb19SktjZyX66OjnIrn2dDc3Ptrooq9YilrU17yWWGe/6LN8ZsB36b4pxHAfiqtfaXjDHrgP8GhAEX+GFr7fWVC1Vkebx/+TL/6+238d+8iQ+41dTEZx9+mENbt3od2uLN2t+XSqcACAVCquwoUuM0PkutuZTN8qdvv83wlSu0GcNVn49De/fy+T17qvMk6l3EIxklZjKvhfxLzwM/Za3dD/QA/9IYcwB4Hvhra+0e4K9Lt0Uq2pnr1/mLV17hBycn+clQiH8RCvEjxvDia6/x3qVLXoe3NNN6o/X2QW8f6pEmsjZofJaakR0f57++/DKPXrvGv2pv58fb2/nJ5mauvP8+3+jv9zo8kVVzz+TMWnvRWvtm6foN4ANgG/B9wNdKD/sa8P0rFaTIcnnp5Em+UF/P9mmNzDsbGvh7gQAv9fdjrfUwuvsTj/SWL73vt5G5XOyRJiK1SeOz1JLjg4N037rFo62t5ToCwbo6nm1v593Tp8mOj3sc4fKKp0KEhrIknKNehyIVZlFzxMaYMHAIeA3YbK29CMUBAti03MGJLLfzly+zb569ELsCAa5dv8745KQHUS2/qQSNkRHNoImsARqfpdqdv3yZfQ0Nc44HfD62W8vHIyMeRLWCYjHcF7oIZfMknKM4ruN1RFIhFpycGWOCwB8CvdbaBf8/xBjznDHmuDHm+JXR0aXEKLJsGvx+RguFOcfHrAWfr3bWtDtOsXm1z0c4FPY6GhFZQRqfpRY0NDTMOz4D3AQaZjU3rwnTErSU26cETYAFJmfGmHqKX/y/a639o9LhS8aYLaX7twCX53uutfar1trD1trDG5ublyNmkSU7uGsXr2Szc46/duMG+3bswF8jyVm0u49UJ0R2H1EFR5EapvFZasXBri5em5hgctb2gnO5HNmmptrtlReL4R4/QiQNqdPHcFynfJG16Z6/RI0xBvhN4ANr7S9Mu+t/AF8uXf8y8CfLH57I8oo++CCn29v54+vXOZfLcT6X439mMrzZ0sLnu7u9Dm9ZhUKdSsxEapjGZ6kl+zduJBQO87VMhpOjo6THxzk2PMwfTEzwvY89Vtv9bKNRnP4eIkN+3P4+XDdFyu3TtoQ1aiHNIz4F/CjwrjEmVTr208BR4A+MMT8GDAA/tDIhiiyfloYG/tmRI7w+OMhfnj+PtZY9u3fz4zt2EJxnrbuISAXT+Cw1w2cMz0YivLNlC6+ePcutsTG2bt3KPw6H2RxcA02aS61xcN3izWfSpEjjuI5OtK4x90zOrLXHgDudrvjc8oYjsvKa6uv59K5dfHrXLq9DERFZMo3PUmt8xhDp7CTS2el1KN6IRstXnWSScCzjXSzimRptu762DedyvDYwwMClSzTU13Nw504+sXlz7RS7EPFYJp3Bfc9lZGSEtvY2wgfChDprdD+EiCyb8clJ3rxwgQ/On6dQKLB72zYe7+qiub7e69CkUjgOUGxSnSHnbSxVaHR4lIH3B7iSvkJDYwPhfWE27dqE8VXPslglZzXm4o0b/M6xY3xibIxvDwS4VSjQd/Ei73V18cXHHquZghciXjn37jneOv4Wvp0+Gnc2kslkcP/C5dEnHmXHQzu8Dk9EKlQunyf56qt0DA3xdCBAHfDu22/z6x99xD85coTQtP6bsgY5DtHuPtye4s2MP08o1KUljYuQSWd4+S9eJt+Zp3FHIyO5ES68doGwGybybZGqSdCUnNWYP3v7bb5jcpJPTKtqtLepid8ZHOStLVt4vKvLw+hkNcRTCVIH82geZ/nlsjne/tbbBD8ZxN9U/PpsXNdIvjNP6tUUm8ObaWxp9DhKEalEf3f2LFuGhvh7oRCmVNxiV1MTLw4P883+fn4oEvE4Qll1jjNzj9mGPJHdR8p3KzFbOGstbzpvYvYb2jvby8ebtzTjvuqy7dw2Nu2qjpaPSs5qyNXRUUaGhni4vX3GcZ8xfDIQ4EXXVXJW4+KpBIkDIxBsIxaJeR1Ozbl05hKFjYVyYjbF3+ynsKHA5bOX2X5wu0fRiUgle+fMGb4cDJYTsylPtrbyCwMDTDz8MPW12MtL5pdMEn52EKLFYicZIBJW+5ulujF0g5GJEdo3z/wNbHyG+h31DJwcUHImqy+XzxM0Zt5ys8G6OsYmJjyISlaN4+B056Ctjd6eXq+jqUkT4xNwh4kx22CL94uIzCM3Pk5rS8uc443G4CsUmCgUlJzVumSyfDX87CCZoJ9I+PaMqRKzpcuP5zENZs7JD4C6QB1j18Y8iGpplJzVkI0tLWT8fobzedr9M//Tnrp1i65t2zyKTFZNIEAooAWNKyW0KQQfFpdPTB8ArLWYa4b2h9vv8mwRWcu2b9rEh1eu8PCssvCDY2M0t7bS5NdPspqWTBL60gCU2/b46Y0+72lItSS4LojJGibHJ6lrmHmSY+zyGJu2VMesGSg5qykNdXU8sW8ff/jOO/xwezvBujqstXyUy/FqXR2xcNjrEGUFxFOJ4pUQpIJZQtpttmLWd61nXcM6rvdfp21vG6bOYCctwx8Os6FpA+u2rfM6RBGpUEf27uWPL15k/dgYWxuLU/BXJyb4k1u3ePqRR+Y94y+1Ix7JaGXLCmpoamD3/t30v9VPe6Sdusbib+DRC6M0XGlg+2eqZ8uBkrMa85kHH6RgLb988iSbJicZtZZCays/9OijbJpnOYVUsVJlp9TBPJTOxIYCXdprtoKMz/Dkdz3JOy++w4UXL0ALcBO6tnbxyHfqx5WI3NkDHR184amn+P2336Z5eBg/cK2+nk8/9hiHtm71OjyRqtf9ZDc+4+PUsVMUWgrYMUuoOcSj3/0ogWD1VENVclZjfMbwuT17+NSuXVy8cYNGv58t82xAlirlOMRDqeLV7ly5spPWqa+exuZGHv/Ox7l14xa5GzkCrQGaWpu8DktEqsDBzZvZ//nPc+HGDQrWsrW1VfvM1oBysS7avA6lpvnqfHQ/1c2Dhx4kez2Lv8FfXO5YZb+BlZzVqIDfz66ODq/DkOVUquyUCfohEAACRDojSsw80tTapKRMRBatzudje7v2p64VU4lZaNMOrWxZJfWBejq2VO9vYCVnIpUsmSyuUweSz2ZLlZ16lJCJiIhUiza1t5GFU3ImUqmmKju1TS2DCGojsYiIiEgNU3ImUkHiqQROZw6A1JfGVdlJREREZA1RcibiNccBIB5KFTcMt7URChQL4msZhIiISJVyHJzuHFA9lQLFe0rORDwUTR+FSPFLOxXMEtqgDcMiIiJVb6rdTSdEOiNeRyNVRMmZyGoqzZIBxS/tDXlCG4pNo0OElJiJiIjUgHJipiJeskhKzkRWS+ksmlvMxcj41aNMRESkVoVCnRrjZdGUnImspKmZMtct9yiLhHvKd+tLW0RERESmKDkTWSmlptH4/dADmYB6lImIiIjInSk5E1lOyWT56lSPsqmNwErK7i0/nmfgvQHOnjzLxPgEm7ZsYndkN20b2+79ZBEREVkR1lounrrIR+99RHYkS1uojd0P72bTrk0YY7wOr6YoORNZJvFUguSzWQgGyZCDgHqULcbkxCR9/7OPK/YKzfua8Qf8DF4cZPBPB/nUt3+K9dvXex2iiIjImtT/Sj8fnP2Apj1N1HfXM5wZ5pVjr3Dw2kH2HN7jdXg1RcmZyP0ozZTFI5lyjzLNlC3NhQ8vcCV/hdDhUPksXNsDbdxqvcVbf/cWn/vi53R2TkREKl48lSB1YJyQ14Esk+y1LCc/PEnoSAhfvQ8Af5OfxnWNvH/sfbr2ddHU2uRxlLVDyZnIEkXTR0n9o3xxTxkQCqlH2f0YOD1AYGdgTgIW2BBg+INhsleztG5o9Sg6ERGRe4unEuWTtbXym+DS2UvQSTkxm1LXWIfdaBk6N8T2g9s9iq72KDkTWSjHIR5KFa925ko9yrpq5svXa5OTk5i6uTNjxhhMnaEwWfAgKhERkYVzOnOENtXWydrJyUmou8OddWh8XmZKzkQWwnEIHz5GJuiHQAAIEOmMaOniMtqyfQvvXXiPwPrAjOMT2QnqJ+oJrg96FJmIiMgCOA5EAvd8WLXZsG0D9qTF7rEzVrfYgoUr0PFEh4fR1Z5VTc4ujF4mnkrMOBaPqGCCVKhkkngkU7x6OFvuUaaEbGXsOLCDM/1nGPlohNZwK6bOMD48TvadLI8+9ih1/judthMREZGV0rG1gy2hLVx8+yJt+9uoa6wjfyvPjfdvsGPLDm05WGarmpxdDkKiZ9qBkRFIJZSgSeUp9Si7PVMW1EzZCmtsaeTpv/c0J14+wcW/vQh+aPI38fhjj9O1v8vr8ERERNYkYwyHv3CYk986yZljZyj4C9QV6tjfvZ89j+9Rsa5ltqrJ2aaWTTzX81z5djKVJOEbxEkfJZq+v2ngeCoEsdh9Rihr2fRZ3cSXipt5VQp/dTW3N/PEdz3BRG6C/ESeQEsA49OXvoiIVDjHIdrdRyqYJ1QzdRpv8zf4eejIQ3Q/2c34rXEamhqoq9eKlpXg6Z6zWCRGMpUkFcyQCt/fayUODJBJJpWgyZJMr65UpMTMS/WBeuoD9V6HISIicm9TiVknRMJHanqVTV19HU31Kpu/kjwvCLJc1WwSfQlCXxogkj5636/l9PdANHr/QUlFi6cSOJ05gGI/khqrriQiIiKrJBAgFArVdGImq8Pz5Gy59Pb0kkwlcTfd3+tkMmnCwWO4zrSDStRqg+OUr0a7+0gdLJbCBwixfCcKRERERESWomaSM1ieH9eO65By+wj39BHOBSCXw0m6Wi5Z7UpLDorFPSiuCVePMhERERGpIDWVnC2HqeloN+PiUppJe3YQV/vZqs+0mbKpHmWhUHGTbiSkyosiIiKyDFwXtyfndRRSI5SczWP2j/ap/WwqOFJFSqXw8Rf/iWcC6lEmIiIiyyyZJPSlgWKFZ63GkWXg8zqAatDb0wttbYS+NDCnibZUEMcpXqb1KAt39xDu7lFiJiIiIsvLcYg+k1brHVlWmjlboN6eXhJ9pXLrapxdceKpBMnDWfD7yUSBQFBflCIiIrKyAgFCgdrraybeUXK2CFMVIRMMKEGrBMkkAPFIptyjLNIZAeYuTRURERERqXRKzhZpqnF2wjeIkz6K0/m81yGtSfFUguSzWQgGyZAjFFKPMhERERGpbvfcc2aM+S1jzGVjzIlpx+LGmI+NManS5btWNszKEovECG3oIrUhTzR9dEZVQFkhpb1kJJNE00dJHMxCVxfhcIRIuEeJmYisSRqjRURqy0JmzpLALwO/Pev4L1prf27ZI6oSsUis2BONY0S7+4r5mZpVr4xSj7LUkXy5+mIopB5lIiJojBYRqSn3TM6stS8ZY8IrH0r1mdrXlPL3EQ4ew3VQgrbcHKfcoywSPqK9ZCIi02iMFvHI1InjYJ4QKggiC5fou3vl9/sppf+Txph3SksqOu7jdapaNBwlEu4plm0/fKxcpEKWQTI5LTFTKXwRkUXQGC2yUqYSsw15Qhu0kkcWLtGXgJGRuz5mqQVBfhX494At/e/PA/90vgcaY54DngNo39y+xLerbOUZNLeP8LODuGpWvTzCYWDQ6yhERKrNgsbo6ePzjvbaHJ9FVkwgQGhDSImZzMtxHVLp1JzjoaEs7gs7MJy743OXNHNmrb1krZ201haA3wCeuMtjv2qtPWytPdzc3ryUt6sK0XCU3ujzZDYECT87qBm05RCN4h4/QiQNqdPHcFzH64hERCreQsfo6ePzxubaHZ9FRFaT4zqk3D5CQ1l6+5hxcV/ouucEzpJmzowxW6y1F0s3fwA4cbfHryWRzghubm6mLEsUjeI4FJcPcAw34+oslYjIXWiMFhFZXVMJGQD5PJEhP07/kbm1KCL3fq17JmfGmN8DosAGY8wg8DNA1BgTobhkwgV+YuHh174MOeKRDHGvA6kVMxK0QRzX0f4zERE0Rot4wnVxe3JeRyEem1rR5WZcMkODxYTsG53FO8PhJRcJXEi1xi/Oc/g3l/Rua0A0HCWVTpE4MAKpBPFIr9ch1YZSghbu6fM6EhGRiqExWmSVJZOEnx0kE/DTq5U8a1YylSSTSRMiALkckYwfp/N5iN3/ay+1IIjcRW9PL8lUkgQDStCWWz6Pm3G9jmLJrLVcv3CdywOXAdi0YxMdWzswxngcmYiIyNp2KZvlrcsXyebH2d22noc2bqS+ru72AxyH6DNpMhuCRDojWsmzxkzNlKXSKRgZoff9NuKpUhuFZSwEqORshcQisWKC5hvESR8tZtNyf6JRYqkUiYYBEn0JenuqK+mdzE/y5jff5OOhjzGdxWSs/2/62bZhG499x2P46u6ns4WIiIgs1V+ePc3vXzyB2WKoazH82eVTdLlt/NQnnqKjqan4oGiUaCqFG8ziZvrI+POk0qmq+z0ii5foS0A2SyjvJ5TPEzvdVpx8WcAessVScraCphK0FINE00dx+nvUpPo+xSO9kEqQOJglmUpWVXGQs6mznL95no5PdWB8xeTMPmg5/8Z51qXW8eBjD3ocoYiIyNrz0bVr/N7lE2x7vJX6+tJM2Tb4+PwIyf4U/+rQU+XHxiO9xB2neD1U3MaS6EsQ6Zz5K10zatVpvsrgUzNlma/vKLV5YkV/zys5W2EzErTuPhwHJWj3KZ6J4GRTuBu8jmThrLWcfu80rY+1lhMzAOMzBPcFOZ06reRMRETEAy9ePEfD9rrbiVnJ1q5W3jl/iaHRUTZMbzdR+h0XJ0o8mST6TBo3M3NPfMLtIxLuUZJWRcol8GfVeolkwPnGjlXrYazkbBXEIrHif3COKUFbLrkcmVzG6ygWzBYsuVyOUDA057761nqGR4exBTsjcRMREZGVlx7L0tJSP+e4MYa6ZsPI2NjM5Gy6WAynNJNW5rqEnx0k5faV98mHQ2ElahXIcZ3yf6NMJk0om8c9fmTuA2PRVYtJydkqmfo/ZMrfRzh4DNdBCdpSRaM4yeIXX8I5Sm+08vfzGZ+hpbWFsetjNHY0zrhv/Po4re2tSsxEREQ8sKs5xJmR67S3B2Ycn5wsULgJ66f2nN3JPL/nXMch2t0HmeKJ5FTwWPGhStAqRjKVLJbAzwaLB3LM35tslSk5W0XlBM3tI3z4GG7SXbUp0poTiq9DOAAAHhhJREFUi+Emk4T/Yboq9p4ZY9j3yD6Ov3Oc+sfr8dUXi38UJgrc/OAmjx963OMIRURE1qbPbAvzzXfPMLp+gubm4gyatZbzZ0Z4un0H7YHAPV5hHqUWQOWb3X2kODaj4nSl/3aZbmqGqZpink8ylSxfL/cm65+2X7ACJk6UnK2yaDhKNBwl4Rwl/OwgbjKpBG2pwmHCuQyu13Es0PaHtnNz5CYnXzoJpf1yZshw4MABuvZ3eRuciIjIGrW1tZX//YEn+PU33+BKx01oAHsNDjV28qWHHl76C0/7oe8QJZo+CkNpAFLrxqvi5DJM24uVzZPIVMeKpfkkU0kylweIXGsAIJoOFgvNdXoc2CzGWrtqb7Z131b73K8/t2rvV+kSfYnb1V+UoC1eaclAqpOq2nR7a+QWVwevYoxhXdc6mlrvsVxCKprjOsVKThVo+N8Ov2GtPex1HFL5Dm/dao8/p/FZ1rZbExP0Dw2Ry+fpamujq61txfqQ/v/t3W1sXNd95/HvIYfi04gaihJFPdG0ZNmyYytj+SGyrCjjuMmmybbpInG6CVJk2kXdAl0g3L6pdt+UKVBAWHR3580iRbLNTnYRtwmUJummhZEg9oWtpHQs2zeyJTMSbY0pShpJjDSihuKIHPLsi5mh+CiNyCHvvcPfBzBI3qHJ/8EV58xvzlOPmyDx0DC0tExd89OW/DP6tlyOaBqcFzsKB3CHQ7CY0USvlc4m88H5w+ZrX1uwf9bImYe693WT6E0Q+dIA3Tqs+u4VpwzE6MXNB2cud2NLI9se0khZNZj+bmK8P+x1OXN8zesCREQCpLGujkc3b16R31U6GqiktCW/HwLa1FqsoRCxdAPQQE8mCvEYqWSSnmhwNmSbyR/B7E4UzjxWCmiySKWANm0udxCmCEjwJN3k3B1Cc7lbOzv5YJ76bF/7keKZiIhfTQ8KPckkkS8NzPuaMNoRXfY3n2f0cdlsYS1Wx6G5U/7icXqWtRJROJPgmxHQBgMzh1uCY+67iCXFdxN9GMxERCRA4nEy84xKJe/L4uYKZ6gtV0ArLbPpPlmaYhkOxAhTtVI4k+pQCmjRYB1OLf5V9ruIIiIilTDPqFSP49D1+FHcVO/UGrBKjKTNGKHz0VosUTiTapPL3fl7RO5g9ruIPe42bdojIiIrLxYj5UBPpBDMnI4cbnZpyzjm9nHamM5PFM6kesRixFyXxPoB3yyqleBI9CZuhfuxsZnvIkYX/v9ERESWVSxGD7FbX6YP4zJIwjkMQCTScdugVtq8qiSSzZM6Mi2QqY/zFYUzqSql3Y8SDw1r7ZnckZNyAKZ2XEwdmbaLpt5FFBERH3I6DkEyCUBPNEPioQGSbpKuSNe8368+LlgUzjyWcA4TyeYL06b0zkVF9ES7cdKHSbV7XYn41dQW+BQ295jqtNRZiYhIEBT7qx4AN0FybJBUZp4t7nM5ohlw+g5APLZy9cmiKZx5KNGb0IvCZTRn23NZstJI0+347ay5+Wq+9S5ipHglor9BEREJpJ5oNz3FkbR5dXVpV+EAUTjzWLw/rBeFy6B0in3COUx37JDX5VSFRG8Cslki+YWfNjKhPG7a9c16v9IW+LNr1ruIIiJSVfRasmr4JpxZa7ly7gpnf32Wmzdv0rapje27t1PfXO91aRJE8TipZLIQ0LQ5yKI5KYdUJlUYhRweJvNCZ+EduIWkUlOhOBLpoCvSteIjaaWagamzyZy+fXO/Ue8iipTtYjbLz8+f5ezoNbY2ruXpzZ1sXrvW67JERKqOL8KZtZaTR09y6swpQp0hatfXcuH8BU6/fZqnP/M0LRtb7vxDRGaLx4m7CRI69+yuTa3JykFXrgFyOZwXy9tqN5VMEvtUGjIZUg29JFK9RLv2rUhIK43uRbNhyOWIpYsHaepsMpFFezN9gf/53i+xWyG8YQ3Hr1/kX97u50+6HuMjW7bd+QeIiEjZfBHOfnP2N5xKnWLd/nXUhGoAaOpoYuTcCG+8/Aax52IYYzyuUgIrl8NJOb5bC+U3M0acMulb6yFLI2XlTgGMx3EcZ+rL0uGZpZ9d6dG0pJsEmBrd6z7ZQk+muLuORsdElmRkbIy/7T9G294mmprqAFi/vpHR9nH+1xtv8mDbRlrqNcNFRKRSfBHOBn49QKgzNBXMSpq2NHHtvWtcH7qu0TNZlJ5MFCfdi5s/Cvhvswq/cFIObv/RwogTQG6Ja7KmhaKUA7HdvVDcRcoNL+3wzOlmj5SVO7onIuV559IlbrZNTAWzksbGOsY2Zjl+Mc2Bzns8qk5EpPr4IpzlcjlCkbmlGGMwDYbxm+MeVCVVIRbDcQrhwKVyoaAalEacYPrarGkjTpWaChiL4RCD4mha4V4Mzvj95d6Tio3uiUhZbuTHMQ123sdqGgzZcfXPIiKV5Itw1tbexuXLl2nc2Djj+uT4JAxDuDXsUWVSFYrhIJY+PBUKVntAS7pJMpcGiF5ZU7wSKhxquZxrs4qjaQ4xetwETkcaAHf9WFn3ZGodXDZPV7bw1OW8qGMoRJbTlvBaGDBw79zH7FXYtkWzWkREKskX4azzwU76v9/P6IZRGtsLAW0yP8m149fYed9O7dgoFeF0HKLHTZDYtzrPP0v0Jm59UVqbFfVmF8vpv7fHTZBgYGZ988lmi6N7B25Nm4wvW4kiAuxqa2PH+6188EGGbZ0tGGOw1nJ+8Drbb67jwQ3acUlEpJJ8Ec4aWxrZ/6n9vPHSG2ROZzD1Bq7Bjh07+NCBD3ldnkjgJXoTU4GswLtgNltPtBvcOwQzAMKFjT60yYfIiqkxhq/u+Qh/9+5bvH3hIjVhw2TW8mDdRv54z15qa2ru/ENERKRsvghnAK2bW3n2i89y7dI1xm+Os7ZtLQ3hBq/LWjalF8ugKSErbnh4VZx9NnukLPOCfzfL8EtQFJG51jU08OePPkU6m+XK6Cit9zTQEQ5rF2URkWXgm3AGYGoMkY6I12Usu+mjGHpRurJKozSJh6o3oE1fmxXvL6zX7HH9G8xEJBg6wmE6wloDLiKynHwVzlYTBTPvTAW0fV5XUjlOysFNu4UvcrnCLobHpq3NinpWmoiIiIiUSeFMJOCmzigbCuG8WNxusatLa7NEREREAkbhTFalHjdC8r5BEs5humOHvC7nriXdJJlMYSt68vniLob7dM6XiIiISIApnMnqFI+TSibp+vxgYNaeOSkHgFQmRebSQGFqrFtcoxmPL+8ZZSIiIiKy7BTOZPWKx4m7CRIBOKYn0ZsorCWjAbJZuvuLaxa1lkxERESkaiicieRyOCmHWFfM60pmKI2UuWn31u6ebgSIaOdFERERkSqkcCarWk8mipPuxc0fBfBNQEs4h4nkCp9H8nlSR4pb4WukTERERKRqKZytMCflQDYL6KwYX4jFcByI7e7F5SipTIp4NL7iZTgph1QmBUAmky5shX9kW2HXRdBGHyIiIiKrgMLZCkq6STJDg0SHQvRkNATiG7EYDjFi6cO4DJJ0kysa0KYOjc5BV64BcuD0HVAgExEREVll7hjOjDHfAv4tcMla+3Dx2nrgu0AXkAK+YK29unxlVodMLnNry/MqPYPqYjbLv545w+ClS9SvWcOeri4e27qVUE2N16XdkdNxiB43QSKaXvY1aEk3OfV5KbA7fdNOxa7Sfx8iUlnqo6VcI2Nj9A4McGpwEGst92/bxkc6O1lbX+91aSIyTTmvmJPAp2ZdOwT8zFq7C/hZ8WspQyzdULUvvN+/epVvv/wyG95/ny9MTPBsNsvpY8f4zuuvk5+c9Lq8svS4kcKOiMuoNILalcrQlZoV2Ev/iYiUJ4n6aLmD4Zs3+earrzL6zjv87s2b/N7YGOMnTvDNV14hk8t5XZ6ITHPHkTNr7SvGmK5Zlz8LxIqffxtwgL+oYF0SMNZafvzmm3wuFGJnYyMA7cC9DQ383/Pn+VU6zWNbtnhbZLmyWdy0W9GRsxkjZaUzykpTW2MxnVEmIouiPlrK4fT388j16zwbiUxd21JfT9O1a/zs17/mcx/+sIfVich0i51rtslaewGg+LG9ciVJEF3IZqnNZtnRMHPUyRjDk/X1nBgY8KiyuxSPE+8Pw/Bw4WyxCkj0JshcGqCrL01XX7oQzKLdGiUTkeWiPlqmWGt558wZPrJ27ZzHngyHefeDD5i01oPKRGQ+y74hiDHmeeB5gHWb1i33rxOPjE1M0EghjM3WVFPD2Pj4yhe1SD3RbnqSSSLxxa89mxHsSmeURbsLX2uUTER8YHr/3LlO/XM1G8/naZxn7Xd9TQ12cpJJa6mZp/8WkZW32JGzi8aYzQDFj5cW+kZr7TestY9bax9vWte0yF8nfrc5HOZSbS3X8/k5j72by3FPR/ASyWLXniWcw0SGsnT3QncvZF7ovBXMRESWX1l99PT+eWOT+udqZYyhc9Mm+m7cmPPY6dFROjZsCMSmXSKrxWJHzv4J+ApwuPjxRxWrSAKpPhTiyd27+d7x43yupYVIKMSktRwfGeF4fT3Pb9/udYl3L5vFTfUCtz+c2kk5uGm38EUud+uMsni8cE2nJojIylIfLTMcfOABfvjKK4RzOTrr6zHGcDaX45/HxvjM7t1elyci05Szlf7fU1hYvMEYMwj8JYUn/O8ZY/4DMAA8t5xFSjA8s3MntTU1fKOvj7UjI4wAkQ0b+IMPf5h1Dcu7A2LFxeOkkkm6Pj9424A2dUZZNl9Yq0ZDYaMPnVEmIitAfbSUY+f69Xx6/35+9PbbMDyMASaamvjkU0/xwIYNXpcnItOUs1vjFxd46NkK11LVEs5hItk8Pe62qh1JMcbwsR072H/PPQzduEFDKERrcefGQCoFtHhmxuWkmySTSRe+yOeLW+Ef0OYeIrLi1EdLuR5sb2f3xz/O5Rs3sNaysblZ68xEfGjZNwSRwuYQc6a6VbG62lo2z7MrVDUonVEWHQrhvFhcR9fVpWAmIiK+Z4yhvbnZ6zJE5DYUzlZCLrdqglnV6eqiK92Lmz9amN44NnZr58W418WJiIiISDVROFtmTsrxugRZilgMxwFSqVvXFLJFREREZBkonC2j0mYR0TSFqW8STJqyKCIiIiIrQOFsmcxYm9S3Ty/wRURERETktnTq4DJQMBMRERERkbulkbMKmxHMOg5Bh9cViYiIiIhIEGjkrIKSbpLMpQG63wkXgpmIiIiIiEiZFM4qxEk5ZDLpW9usi4iIiIiI3AWFswqK0ECPG/G6DBERERERCSCtOasQN+0SyWYBhTPxp3Q2yyunT9N/7hzGGHZv387Hdu1ifWOj16UFyqS1vD44yOunT3M1m6U1HObJ++/nia1bMcZ4XZ6IiATMyNgYr7z/Pm+fOcPY+Didmzbx0fvv597WVq9LC5zTv/kNR0+d4tzlyzSsWcOeHTs4uGMHDaHgRJ7gVOpjid4EkaEsqSPbdECx+NK54WG+8+qrfGxigt9pbmYCePPMGb514QJ/dPCgAtpd+H8nTnDl1Ck+29TE5rVrOX/zJj/95S+5+MAD/M6HPuR1eSIiEiCj4+N86xe/YOfVq/xxOExzfT19ly/z/XSazzz1FA+2t3tdYmC4Fy7w8muv8W/q6rgvHOb6xARHT5wgefEif7R/P2tqa70usSya1rgETsoh4RxWMBPf+9m77/KJyUk+0tJCY20t4dpaDq5bxxO5HK/093tdXmBcuH6d9/r7+XIkwvaGBkLG0NnQwB9EIpw6fZpLIyNelygiIgHy+uAg265e5dOtrbTW1bGmpoY94TDP1dfzk+PHmbTW6xIDIT85yU9/9Su+1NTEQ83NrKmpoa2ujt+NRFg3NIR74YLXJZZN4WyRnJSDm+olks0rmImvjU1McDad5pFweM5je5ub6Tt71oOqgunXQ0PssZa6mplPnWtqanjYWvouX/aoMhERCaK+s2fZO8/slc76empGRvSmX5kGh4dpvXmTTWvWzLhujGFvfT19g4MeVXb3NK1xEWYEs2MHIB7zuiSRBVlrwdp534mpNYbJyckVrymoJq2lZoF1ZbXFx0VERMplraV2nn7FGEOttepXyjS5wOscCN5rHY2c3SUn5eD2HyWaphDMYjGvSxK5rfpQiI6NG+m7cWPOY8dHRti1dasHVQXTfW1tnAAmZnWWE9Zywhh2tbV5U5iIiATSfVu3cnx0dM719NgYo42NbGpu9qCq4NnW0sLlUIir4+NzHjueywXqtY7C2V1IuslCMBsK4fTtUzCTwHjmoYf453yekyMjTFpL3lrevH6dV2pr+eiuXV6XFxjbW1rYsH07RzKZqQ7gyvg437t6lY577mFrS4vHFYqISJA8uX07fc3NvHrtGjcnJ7HWcmZ0lO+OjPDMI49QW6OX6uVYU1vLwUce4YXr1zmby2GtZXRigpeuXePsunXs3bLF6xLLpmmNZUq6STJDg4Vg1nEIOryuSKR8O1pb+dzBg7x88iQ/uHwZjKFz82a+/OCDdMyzFk3mZ4zhuUcfxWlp4Zv9/UyOjFBTX89je/bwsR07vC5PREQCZm19PX944AA/7evjb86epWZyknWRCM/u3cvDmzZ5XV6gPNXZSUMoxA/7+sheu4atrWX3vffyhw88QGNdndfllU3h7C5Es2GcvqiCmQTSjtZWdjz9NDfzeYwxgdlS1m9CNTX81q5dfPy++8jl8zSEQguuQxMREbmT1sZGvvDoo4zv2UN+cpKGUEjnZi7So1u2EN28mVw+T11tLaEAjjwqnN2NXM7rCkSWrD5ABzH6WY0xNAXonTgREfG3utpa6vTG6ZIZYwI1UjabXqWVIdGbgOFhYukWrTMTEREREZFlEbyxvhVWCmbdJ1voiXZ7XY6IiIiIiFQpjZzdRimYZV7o1CHTIiIiIiKyrBTOFpBwDhcOmT6iYCYiIiIiIstP4WwWJ+XgpnqLwWybgpmIiIiIiKwIhbNZUplUIZgdOwDxmNfliIiIiIjIKqENQebRlQ1pV0YREREREVlRCmfTOCmHzNCg12WIiIiIiMgqpGmNRUk3SWZokOhQCKdvH3R4XZGIiIiIiKwmCmfMCmYdhxTMRERERERkxa36aY1JN0nm0gDd74QLwUxERERERMQDq3rkrHTIdPfJFnqi3V6XIyIiIiIiq9iqHTlTMBMRERERET9ZlSNnpWCWeaFTh0yLiIiIiIgvrLqRs0RvgshQVsFMRERERER8ZdWFM4B4f1jBTEREREREfGVVhTMn5UAu53UZIiIiIiIic6yacOakHNxUL5Fsnp5M1OtyREREREREZlgVG4I4KQe3/2jhkOm+AxCLeV2SL1hr6b9yhZPnz5OfmKCrvZ09mzZRV1vrdWkiIiKr2sVslrfOneP6jRu0RyLs3bKFtfX1XpclIstsSeHMGJMCrgMTQN5a+3gliqqkpJskMzRYDGb7FMyKJq3liOvymw8+YG9NDfU1NZxMpfhFJEL8qafUAYiIBFwQ+miZ378ODPDzt97icWBrbS0DAwN8/d13+cL+/XRFIl6XJyLLqBIjZ89Ya4cq8HMqTsFsYW+cO8eNM2d4vrWVWmMAiAIvXbvGv5w4we/v3ettgSIiUgm+7aNlfpdGRvj5W2/xfHMzLaHCy7RHgAdHRzny2mv8p098gtqaVbMqRWTVqdq/7lIw634njNNxSMFslrfef5+DjY1Twazk6bVrOTM4yI3xcY8qExERWb3cc+fYC1PBrGRHYyNto6OcvnLFm8JEZEUsNZxZ4CfGmDeMMc/P9w3GmOeNMceMMcduXLuxxF9XHiflkMmk6X4nTE+0e0V+Z9BkR0dpq6ubc72+poYmaxlVOBMRCbrb9tHT++fLN1amf5Y7y46O0rbA2u82YGRsbGULEpEVtdRw9rS1di/w28CfGWMOzv4Ga+03rLWPW2sfb1rXtMRfV74IDfS4mpe9kPbWVlLzHCuQyecZDYVo0ZozEZGgu20fPb1/3ti0cv2z3F77unWk8vk51621fAC0NzevfFEismKWFM6steeLHy8BPwCerERRS+WmXchmvS7D15667z5eGh/nyrQRsrHJSX58/TqP3X+/dmwUEQk4v/bRcnuPbtnCqfp6Tk0bzbTW4gwP09jezraWFg+rE5HltugNQYwxzUCNtfZ68fNPAn9VscoWKdGbgOFhUkc6IR73uhzf2rl+PR994gm+4bp0joxQD7xnDLvvv59ndu70ujwREVkCv/bRcmfNa9bw7/fv58jrr/NqJkObMQxYS3N7O7//2GOYWWvFRaS6LGW3xk3AD4pPEiHgBWvtixWpapESzmEi2byCWZke37aNRzo6OH3lCvnJSZ6NRIg0NHhdloiILJ3v+mgp3/Z16/jqs8/y3pUrZMfGeKK5mS1r1yqYiawCiw5n1tr3gQ9XsJZFc1IObqq3GMy2KZjdhfpQiIfb270uQ0REKshPfbQsTo0x7Gpr87oMEVlhgd9KX8FMRERERESqQSUOofbMjGB27ADEY16XJCIiIiIisiiBHTlzUg5u/1GiaQrBTIdMi4iIiIhIgAUynJVGzKJDIZy+fQpmIiIiIiISeMZau3K/zJjLwAdlfOsGYGiZy1lJ1dSeamoLqD1+Vk1tAW/ac4+1duMK/04JoLvon6G6/jarqS1QXe2pprZAdbWnmtoCPuufVzSclcsYc8xa+7jXdVRKNbWnmtoCao+fVVNboPraI6tXNf1brqa2QHW1p5raAtXVnmpqC/ivPYGc1igiIiIiIlJtFM5ERERERER8wK/h7BteF1Bh1dSeamoLqD1+Vk1tgeprj6xe1fRvuZraAtXVnmpqC1RXe6qpLeCz9vhyzZmIiIiIiMhq49eRMxERERERkVXFd+HMGJMyxrxtjHGNMce8ruduGWO+ZYy5ZIx5Z9q19caYnxpjThc/tnpZY7kWaEuPMeZc8f64xphPe1ljuYwx240xLxtj3jXGnDDGfLV4Paj3ZqH2BPX+NBhjfmmM+VWxPV8rXr/XGPNa8f581xizxuta7+Q2bUkaY85MuzdRr2sVuRvqn/2jmvpnqK4+Wv2zfwWlf/bdtEZjTAp43FobyPMTjDEHgSzwf6y1Dxev/VfgirX2sDHmENBqrf0LL+ssxwJt6QGy1tq/8bK2u2WM2Qxstta+aYxZC7wB/B4QJ5j3ZqH2fIFg3h8DNFtrs8aYOuAo8FXgz4F/tNb+gzHmb4FfWWu/7mWtd3Kbtvwp8GNr7RFPCxRZJPXP/lFN/TNUVx+t/tm/gtI/+27kLOista8AV2Zd/izw7eLn36bwR+p7C7QlkKy1F6y1bxY/vw68C2wluPdmofYEki3IFr+sK/5ngY8DpSfLQNyf27RFRDyk/tm/qqmPVv/sX0Hpn/0YzizwE2PMG8aY570upkI2WWsvQOGPFmj3uJ6l+o/GmOPFaRW+n2IwmzGmC3gUeI0quDez2gMBvT/GmFpjjAtcAn4KvAdkrLX54rcMEpAObnZbrLWle/PXxXvzP4wx9R6WKLIY6p/9L5DP/9NVUx+t/tl/gtA/+zGcPW2t3Qv8NvBnxaF78Y+vAzuBKHAB+G/elnN3jDFh4PtAt7V22Ot6lmqe9gT2/lhrJ6y1UWAb8CTw4HzftrJVLc7sthhjHgb+M7AbeAJYD/h6ao7IPNQ/+1tgn/9LqqmPVv/sT0Hon30Xzqy154sfLwE/oPCPIOguFucgl+YiX/K4nkWz1l4s/sOeBL5JgO5PcX7x94HvWGv/sXg5sPdmvvYE+f6UWGszgAPsAyLGmFDxoW3Aea/qWoxpbflUcaqLtdbeBP43Abw3srqpf/a3oD//V1Mfrf7Z//zcP/sqnBljmouLJzHGNAOfBN65/f8VCP8EfKX4+VeAH3lYy5KUniSL/h0BuT/FRaB/B7xrrf3v0x4K5L1ZqD0Bvj8bjTGR4ueNwG9RmKf/MvD54rcF4v4s0Ja+aS8wDIW5+YG4NyKg/jkIgvr8D9XVR6t/9q+g9M++2q3RGLODwrtxACHgBWvtX3tY0l0zxvw9EAM2ABeBvwR+CHwP6AQGgOestb5fyLtAW2IUhuQtkAL+pDQf3M+MMQeAV4G3gcni5f9CYR54EO/NQu35IsG8P3soLCiupfCm0festX9VfE74BwrTDN4Cvlx8Z8u3btOWl4CNgAFc4E+nLUwW8TX1z/5STf0zVFcfrf7Zv4LSP/sqnImIiIiIiKxWvprWKCIiIiIislopnImIiIiIiPiAwpmIiIiIiIgPKJyJiIiIiIj4gMKZiIiIiIiIDyiciYiIiIiI+IDCmYiIiIiIiA8onImIiIiIiPjA/weyvjipu20WGQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1080x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "titles = ('K Neighbors with k=1', 'K Neighbors with k=2')\n",
    "\n",
    "fig = plt.figure(figsize=(15, 5))\n",
    "plt.subplots_adjust(wspace=0.4, hspace=0.4)\n",
    "\n",
    "X0, X1 = X_train[:, 0], X_train[:, 1]\n",
    "\n",
    "x_min, x_max = X0.min() - 1, X0.max() + 1\n",
    "y_min, y_max = X1.min() - 1, X1.max() + 1\n",
    "xx, yy = np.meshgrid(np.arange(x_min, x_max, 0.2),\n",
    "                     np.arange(y_min, y_max, 0.2))\n",
    "\n",
    "for clf, title, ax in zip(models, titles, fig.subplots(1, 2).flatten()):\n",
    "    Z = clf.predict(np.c_[xx.ravel(), yy.ravel()])\n",
    "    Z = Z.reshape(xx.shape)\n",
    "    colors = ('red', 'green', 'lightgreen', 'gray', 'cyan')\n",
    "    cmap = ListedColormap(colors[:len(np.unique(Z))])\n",
    "    ax.contourf(xx, yy, Z, cmap=cmap, alpha=0.5)\n",
    "    ax.scatter(X0, X1, c=y_train, s=50, edgecolors='k', cmap=cmap, alpha=0.5)\n",
    "    ax.set_title(title)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 习题3.2\n",
    "&emsp;&emsp;利用例题3.2构造的$kd$树求点$x=(3,4.5)^T$的最近邻点。\n",
    "\n",
    "**解答：**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x点的最近邻点是(2, 3)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "from sklearn.neighbors import KDTree\n",
    "\n",
    "train_data = np.array([(2, 3), (5, 4), (9, 6), (4, 7), (8, 1), (7, 2)])\n",
    "tree = KDTree(train_data, leaf_size=2)\n",
    "dist, ind = tree.query(np.array([(3, 4.5)]), k=1)\n",
    "x1 = train_data[ind[0]][0][0]\n",
    "x2 = train_data[ind[0]][0][1]\n",
    "\n",
    "print(\"x点的最近邻点是({0}, {1})\".format(x1, x2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 习题3.3\n",
    "&emsp;&emsp;参照算法3.3，写出输出为$x$的$k$近邻的算法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**解答：**  \n",
    "**算法：用kd树的$k$近邻搜索**  \n",
    "输入：已构造的kd树；目标点$x$；    \n",
    "输出：$x$的最近邻    \n",
    "1. 在$kd$树中找出包含目标点$x$的叶结点：从根结点出发，递归地向下访问树。若目标点$x$当前维的坐标小于切分点的坐标，则移动到左子结点，否则移动到右子结点，直到子结点为叶结点为止；  \n",
    "2. 如果“当前$k$近邻点集”元素数量小于$k$或者叶节点距离小于“当前$k$近邻点集”中最远点距离，那么将叶节点插入“当前k近邻点集”；  \n",
    "3. 递归地向上回退，在每个结点进行以下操作：  \n",
    "(a)如果“当前$k$近邻点集”元素数量小于$k$或者当前节点距离小于“当前$k$近邻点集”中最远点距离，那么将该节点插入“当前$k$近邻点集”。  \n",
    "(b)检查另一子结点对应的区域是否与以目标点为球心、以目标点与于“当前$k$近邻点集”中最远点间的距离为半径的超球体相交。如果相交，可能在另一个子结点对应的区域内存在距目标点更近的点，移动到另一个子结点，接着，递归地进行最近邻搜索；如果不相交，向上回退；\n",
    "4. 当回退到根结点时，搜索结束，最后的“当前$k$近邻点集”即为$x$的最近邻点。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 构建kd树，搜索待预测点所属区域\n",
    "from collections import namedtuple\n",
    "import numpy as np\n",
    "\n",
    "\n",
    "# 建立节点类\n",
    "class Node(namedtuple(\"Node\", \"location left_child right_child\")):\n",
    "    def __repr__(self):\n",
    "        return str(tuple(self))\n",
    "\n",
    "\n",
    "# kd tree类\n",
    "class KdTree():\n",
    "    def __init__(self, k=1):\n",
    "        self.k = k\n",
    "        self.kdtree = None\n",
    "\n",
    "    # 构建kd tree\n",
    "    def _fit(self, X, depth=0):\n",
    "        try:\n",
    "            k = self.k\n",
    "        except IndexError as e:\n",
    "            return None\n",
    "        # 这里可以展开，通过方差选择axis\n",
    "        axis = depth % k\n",
    "        X = X[X[:, axis].argsort()]\n",
    "        median = X.shape[0] // 2\n",
    "        try:\n",
    "            X[median]\n",
    "        except IndexError:\n",
    "            return None\n",
    "        return Node(location=X[median],\n",
    "                    left_child=self._fit(X[:median], depth + 1),\n",
    "                    right_child=self._fit(X[median + 1:], depth + 1))\n",
    "\n",
    "    def _search(self, point, tree=None, depth=0, best=None):\n",
    "        if tree is None:\n",
    "            return best\n",
    "        k = self.k\n",
    "        # 更新 branch\n",
    "        if point[0][depth % k] < tree.location[depth % k]:\n",
    "            next_branch = tree.left_child\n",
    "        else:\n",
    "            next_branch = tree.right_child\n",
    "        if not next_branch is None:\n",
    "            best = next_branch.location\n",
    "        return self._search(point,\n",
    "                            tree=next_branch,\n",
    "                            depth=depth + 1,\n",
    "                            best=best)\n",
    "\n",
    "    def fit(self, X):\n",
    "        self.kdtree = self._fit(X)\n",
    "        return self.kdtree\n",
    "\n",
    "    def predict(self, X):\n",
    "        res = self._search(X, self.kdtree)\n",
    "        return res"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x点的最近邻点是(2, 3)\n"
     ]
    }
   ],
   "source": [
    "KNN = KdTree()\n",
    "X_train = np.array([[2, 3], [5, 4], [9, 6], [4, 7], [8, 1], [7, 2]])\n",
    "KNN.fit(X_train)\n",
    "X_new = np.array([[3, 4.5]])\n",
    "res = KNN.predict(X_new)\n",
    "\n",
    "x1 = res[0]\n",
    "x2 = res[1]\n",
    "\n",
    "print(\"x点的最近邻点是({0}, {1})\".format(x1, x2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----\n",
    "参考代码：https://github.com/wzyonggege/statistical-learning-method\n",
    "\n",
    "本文代码更新地址：https://github.com/fengdu78/lihang-code\n",
    "\n",
    "习题解答：https://github.com/datawhalechina/statistical-learning-method-solutions-manual\n",
    "\n",
    "中文注释制作：机器学习初学者公众号：ID:ai-start-com\n",
    "\n",
    "配置环境：python 3.5+\n",
    "\n",
    "代码全部测试通过。\n",
    "![gongzhong](../gongzhong.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7-final"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}